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ABSTRACT 

Selected members of the Rural Education Special 
Interest Group (RE/SIG) of the American Educational Research 
Association participated in a modified Delphi study to examine the 
agenda for rural educational research proposed by the Federal 
Interagency Committee on Education (FICE) Subcommittee on Rural 
Education. The Delphi technique is a procedure designed to secure 
opinions and judgments of experts in specialized fields and to seek 
consensus as to the most likely scenarios, The 28 participants 
validated the contents of the original 1991 FICE agenda topics and 
research and development areas. Through two rounds of consensus 
building, the participants agreed with the content of the original 
FICE agenda's six major topics. In addition, they agreed with the 
content of 31 of the AA original FICE agenda research and development 
areas within each of the 6 topics. They suggested 23 new research and 
development areas, across the 6 major topics, during the first round 
of the study. In Round Two, the members validated the contents of 
these new areas and agreed on 16 of them for inclusion in Round 
Three. During Round Three, participants prioritized the six major 
topics and developed a research and development budget. The topics, 
in order of priority, are the following: (1) overall school 
effectiveness; (2) school-community partnerships; (3) human 
resources; (A) finance and governance issues; (5) use of technology; 
and (6) curriculum provisions. Appendices contain introductory 
letters and instruments for the three rounds and commentary papers by 
RE/SIG members. (KS) 
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Executive Summary 



In 1989, the Federal Interagency Committee on 
Education (FICE) Subcommittee on Rural Educa- 
tion examined the state of rural education research, 
drawing upon the knowledge of specialists inside 
and outside the federal government. This examina- 
tion led to the release of An Agenda for Research and 
Development on Rural Education (hereafter, referred 
to as the Agenda) in 1991 (United States Depart- 
ment of Education[ED], 1991). The Agenda pre- 
sented six major research topics (school effective- 
ness, curriculum provisions, school-community 
partnerships, human resources, use of technology, 
and finance and governance issues) and several 
areas for research and development under each 
topic. 

In 1991, during the business meeting of the 
Rural Education Special Interest Group (RE/SIG) 
of the American Educational Research Association 
(AERA), members discussed the Agenda and its 
utility. It became apparent through this discussion 
that the Agenda was not being accorded much at- 
tention in professional research publications or as- 
sociation meetings. Likewise, there seemed to be 
little evidence that any of the FICE Agenda topics 
were being researched systematically. 

The RE/SIG members present at the business 
meeting decided to focus on the Agenda as a mecha- 
nism for facilitating members' communication 
throughout the year. They surmised that the Agenda 
could provide the group with an opportunity to 
involve themselves in activities that would produce 
reflective commentary that could validate, update, 



and extend the contents of the original Agenda 
developed by the FICE and the ED. 

In response to the members' resolve, an RE/ 
SIG official and some of his colleagues at the Appa- 
lachia Educational Laboratory designed this modi- 
fied Delphi study and conducted it over a 10-month 
period in 1992 with selected members of the group. 
(The collection of data was conducted from Febru- 
ary 1992-November 1992; however, data analysis 
and report writing occurred in 1993-94.) 

The authors conducted this modified Delphi 
study with selected members of the RE/SIG of 
AERA. The primary purpose was to examine the 
FlCE's Subcommittee Agenda for rural educational 
research. 

Secondary purposes of this study were to de- 
velop a menu of researchable issues for rural educa- 
tional researchers; and to draw upon the knowl- 
edge of selected RE/SIG members to prioritize the 
major topics of the Agenda, and to allocate a hypo- 
thetical budget across the major topics. 

The Delphi technique is a procedure designed 
to secure opinions and judgements of experts in 
specialized fields and to seek consensus as to the 
most likely scenarios. The authors decided to use 
this procedure to gather data for this report; how- 
ever, only two of the three rounds could be classi- 
fied as "Delphi rounds." The third round was more 
like a traditional marketing survey. Based on this 
study configuration, the authors refer to this method 
as a modified Delphi process. 

Based on the findings, the authors drew the 
following conclusions. 



vii 



9 



Appalachia Educational Laboratory • Occasional Paper 35 



Implications for Future 
Rural Educational Research 
and Development 

Certain implications for future rural educational 
research and development can be derived from this 
study. These implications are as follows: 



• The RE/SIG members expanded the contents 
of the original 1991 FICE Agenda research and 
development areas. They suggested 23 new 
research and development areas, across the six 
major topics, during the first round of this study. 
In Round Two, the members validated the 
contents of these new areas and agreed on 16 of 



them for inclusion in Round Three. Therefore, 
they expanded the original FICE Agenda by 16 
research and development areas. 

• This research effort produced an updated menu 
of researchable issues available for immediate 
use by rural educational researchers. Building 
upon the earlier efforts of the FICE 
Subcommittee on Rural Education, the RE/SIG 
members added their knowledge and insights 
and adapted the original 1991 Agenda to reflect 
changes in the conditions of rural educational 
research. The Rural Education Research and 
Development Menu should continue to facilitate 
a dialogue on the problems and contributions 
of rural education, encourage research and 
development on rural concerns, and promote 
coordination and collaboration among 
educational researchers. 

In addition to an updated agenda, this research 
effort provided a first attempt at prioritizing 
the six major topics and their research and 
development areas. This priority ranking, along 
with the hypothetical budget allocations, may 
assist rural educational researchers with 
decisions concerning their research and 
development resources. 



• The RE/SIG members who participated in this 
study validated the contents of the original 1991 
FICE Agenda topics and research and 
development areas. Through two rounds of 
consensus building, the participants agreed with 
the content of the original FICE Agenda's six 
major topics. In addition, they agreed with the 
content of 31 of the 44 (70%) original FICE 
Agenda research and development areas within 
each of the six topics. 
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Chapter One: 
Introduction 



In 1989, the Federal Interagency Committee on 
Education (FICE) Subcommittee on Rural Educa- 
tion examined the state of rural education research, 
drawing upon the knowledge of specialists inside 
and outside the federal government. This examina- 
tion led to the release of An Agenda for Research and 
Development on Rural Education (hereafter, referred 
to as the Agenda) in 1991 (United States Depart- 
ment of Education [ED], 1991). The Agenda pre- 
sented six major research topics (school effective- 
ness, curriculum provisions, school-community 
partnerships, human resources, use of technology, 
and finance and governance issues) and several 
areas for research and development under each 
topic. 

In the Agenda, the FICE "recognized a need to 
better examine the state of rural education" and 
"attempted to identify and articulate some of the 
[related] research issues" (ED, 1991). 

Further, the FICE sought to: 

• open a channel for dialogue on problems and 
contributions of rural education; 

• encourage the focus of research and 
development resources on rural concerns; and 

• promote coordination and collaboration among 
researchers. 

The FICE Subcommittee and the ED invited 
educational researchers to use the Agenda to for- 
mulate research studies and to share their findings 
with them. In addition, the authors of the Agenda 
asked the profession to submit comments about 
ways to adapt the Agenda to the changing condi- 
tions in rural settings. 



In 1991, during the business meeting of the 
Rural Education Special Interest Group (RE/SIG) 
of the American Educational Research Association 
(A ERA), members discussed the Agenda and its 
utility. It became apparent through this discussion 
that the Agenda was not being accorded much at- 
tention in professional research publications or as- 
sociation meetings. Likewise, there seemed to be 
little evidence that any of the FICE Agenda topics 
were being researched systematically. 

The RE/SIG members present at the business 
meeting decided to focus on the Agenda as a mecha- 
nism for facilitating members' communication 
throughout the year. They surmised that the Agenda 
could provide the group with an opportunity to 
involve themselves in activities that would produce 
reflective commentary that could validate, update, 
and extend the contents of the original Agenda 
developed by the FICE and the ED. 

In response to the members' resolve, an RE/ 
SIG official and some of his colleagues at the Appa- 
lachia Educational Laboratory designed this modi- 
fied Delphi study and conducted it over a 10-month 
period in 1992 with selected members of the group. 
(The collection of data was conducted from Febru- 
ary 1992-November 1992; however, data analysis 
and report writing occurred in 1993-94.) 

Purpose 

The authors conducted this modified Delphi 
study with selected members of the RE/SIG of 
AERA. The primary purpose of this study was to 
examine the FICE's Subcommittee Agenda for rural 
educational research. 



r 



Secondary purposes of this study were to de- 
velop a menu of researchable issues for rural edu- 
cational researchers; and to draw upon the knowl- 
edge of selected RE/SIG members to prioritize the 
major topics of the Agenda, and to allocate a hypo- 
thetical budget across the major topics. 

Objectives 

The following seven objectives guided the work 
of this modified Delphi study: 

1. To expand opportunities for RE/SIG members 
to interact about research issues. 

2. To validate, update, and expand the FICE 
Agenda by having RE/SIG members evaluate 
their level of agreement with the recommended 
FICE six major topics and related areas for re- 
search and development 

3. To produce a menu of researchable issues for 
rural educational researchers. 

4. To prioritize the major topics and reflect the 
view's of selected RE/SIG members in terms of 
urgency with which the major topics need to be 
addressed. 
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5. To allocate a hypothetical research and devel- 
opment center budget of $5,000,000 among the 
major topics. 

6. To produce a document that communicates to 
educators and policymakers, especially those 
with strong rural interests, the research menu, 
priority list, and hypothetical budget allocation 
derived from selected RE/SIG members. 

7. To keep the selected RE/SIG members involved 
in all three rounds of the study. 

Audience for this Report 

Researchers and policymakers, especially those 
interested in rural education issues, constitute the 
primary audience for AEL Occasional Paper 35, 
Priorities for Research and Development With Rural, 
Small Schools: Results of a Modified Delphi Study 
With a Panel of Rural Researchers. Secondary audi- 
ences include education practitioners working in 
rural areas; teacher education institutions that pre- 
pare educators to work in rural communities; rural 
citizens and others working on community and 
economic development projects; foundations and 
other philanthropic organizations that sponsor 
projects in rural communities; and researchers in- 
terested in applications of the Delphi technique. 
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Chapter Two: 
Procedures Used 



The Delphi technique is a procedure designed 
to secure opinions and judgments of experts in 
specialized fields and to seek consensus as to the 
most likely scenarios. The authors decided to use 
this procedure to gather data for this report; how- 
ever, only two of the three founds could be classi- 
fied as "Delphi rounds/ The third round was 
more like a traditional marketing survey. Based on 
this study configuration, the authors refer to this 
method as a modified Delphi process. 

The overall study process is summarized in 
Figure 1. The data for this modified Delphi study 
were collected over a 10-month period in 1992. 
Data analysis and report writing occurred in 1993- 
94. 



Selection of Participants for 
Modified Delphi Study 

As indicated above, one of the objectives of this 
modified Delphi study was to expand opportuni- 
ties for RE/SIG members to interact about research 
issues. Therefore, participants for this study in- 
cluded members of the RE/SIG only. 

The authors agreed on a purposeful sample of 
participants from the RE/SIG membership list. The 
reason for a purposeful sample instead of a random 
sample was that the authors wanted the group to 
be balanced in terms of geography and gender. 



An Agenda for 
Research and 
Development on 
Rural Education 
(F1CE Subcom- 
mittee) 

• Six Major 
Topics 

• Research and 
Development 
Area within 



Round One 

• 90%SA,A,N 
for Inclusion of 
Original RCE 
Agenda Items 

• Suggest New 
Topics/Areas 
for Agenda 



Round Two 

• 90% SA f A, N, 
for Inclusion of 
Suggested 
New Topics/ 
Areas 

• Suggest New 
Areas for New 
Agenda Topics 



RE/SIG Delphi 
Menu 

• Six Major 
Topics (Same) 

• Research and 
Development 
Areas Within 
(Changed) 



Round Three 

• Prioritize the 
Six Major 
Topics 

• Allocate 
Hypothetical 
Budget to 
Topics 



— November 1992 — 



-1989— 



— February 1992- 



— May 1992— 



Figure 1 

Overview of Modified Delphi Study Process 
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Therefore, 20 females and 20 males from different 
geographical regions of the United States were in- 
vited to participate in this modified Delphi study. 
The authors telephoned each of the 40 RE/SIG 
members, explained the focus of the study, and 
asked them if they would like to participate in the 
modified Delphi study. Thirty of the 40 (75%) RE/ 
SIG members agreed to participate in the study. 
(The reader should note that although 30 members 
agreed to participate in the study, one of these 30 
members did not respond to a followup mailing 
and was eliminated from the study. Therefore, the 
sample size for all three rounds of the modified 
Delphi study was 29.) 

Of the 29 RE/SIG members who agreed to 
participate in the study, 19 (66%) were male and 10 
(34%) were female. In terms of geographical loca- 
tion, one participant was from each of the follow- 
ing areas: Alabama; California; Washington, DC; 
Georgia; Illinois; Iowa; Maine; Maryland; Minne- 
sota; Mississippi; Missouri; Montana; New York; 
Oklahoma; South Carolina; Utah; Vermont; Wash- 
ington; and West Virginia. Two participants were 
from each of the following areas: Colorado, Kan- 
sas, Kentucky, Tennessee, and Texas. 

Participants were asked to provide the authors 
with information concerning their professional role 
and their place of employment. All but one pro- 
vided information about their respective roles. Fif- 
teen were higher education professors, seven were 
higher education administrators, three were re- 
searchers, two listed "other" as their role, and one 
was a staff member of a state education agency. 

In terms of place of employment, all provided 
this information. Of the 29 participants, 24 were 
employed at institutes of higher education; two 
were employed at educational associations; and 
one each was employed at state education agen- 
cies, a Lab/ Center, or the Department of Educa- 
tion. 



Round One 

kerns on the first round of the modified Delphi 
study included the six major topics (The Effective- 
ness of Rural Schools, Curricular Provisions in Ru- 
ral Schools, School and Community Partnerships, 
Human Resources for Rural Schools, Use of Tech- 
nology in Rural Schools, and Financial Support and 
Governance for Rural Schools) and the associated 
research and development areas developed by the 
FICE Subcommittee on Rural Education, (See Ap- 
pendix A, Round One Instrument Documentation.) 
The authors pilot tested the Round One instrument 
with members of AEL's Rural Coordinating Unit 
during January 1992. Minor changes in directions 
to respondents were made to the documentation 
and a revised copy was mailed in February 1992 to 
the study participants. 

In Round One, the participants completed three 
tasks. First, the participants examined the Agenda, 
developed by the FICE Subcommittee, and indi- 
cated the degree to which they believed that each 
research and development area was a priority. Par- 
ticipants used the following five-point scale: 
Strongly Agree, Agree, Neutral, Disagree, and 
Strongly Disagree. 

Second, the participants suggested other re- 
search and development areas for the six research 
and development agenda topics. The authors en- 
couraged the participants to think creatively about 
other possible areas for research and development. 
Third, participants in Round One nominated addi- 
tional topics, other than the six extant ones, for 
consideration in subsequent rounds of this study. 

Once all instruments were returned, the au- 
thors completed two steps. Items from the first task 
were retained in the study if 90 percent 1 of the 
respondents Strongly Agreed, Agreed, or were Neu- 
tral with/ toward the items. These items were filed 
until the completion of Round Two. Next, research 



^e 90 percent criterion was agreed upon by the authors before the study began. The high agreement criterion was 
based on the objective to validate, update, and expand the FICE Subcommittee Agenda. 
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and development areas generated for existing FICE 
Agenda topics, and the nominated additional top- 
ics, were aggregated into the instrumentation for 
Round Two. 

Round Two 

The second round of the modified Delphi study 
focused on both the extant areas for research and 
development under the FICE Subcommittee Agenda 
topics and on the newly suggested topic and areas 
for research and development that participants pro-, 
vided as write-ins during Round One. (See Appen- 
dix B, Round Two Instrument Documentation.) The 
authors mailed Round Two of the study to partici- 
pants in May 1992. Participants returned their com- 
pleted instruments by June 1992. 

Participants completed two tasks in Round Two. 
First, participants reviewed the RE/SIG-generated 
areas of research and development, for original FICE 
Agenda topics, and indicated the degree to which 
they felt each area presented was a priority. The 
same five-point scale used in Round One was used 
in Round Two. 

Second, participants examined the newly sug- 
gested topic, "Unique Aspects of Rural Communi- 
ties,* and indicated the degree to which they felt 
this topic was a priority. Participants were instructed 
that if they Strongly Agreed, Agreed, or were Neu- 
tral with/ toward this new topic, they should nomi- 
nate associated areas of research and development 
for consideration in subsequent rounds of the modi- 
fied Delphi study. 

Following receipt of the completed instruments, 
the authors retained items from the first task if 90 
percent of the respondents Strongly Agreed, Agreed, 
or were Neutral with/toward the item. In terms of 
the second task, the topic was retained if 90-percent 
of the respondents Strongly Agreed, Agreed, or 
were Neutral with/ toward the item. (The topic did 
not meet the 90 percent criterion. If the topic had 
been retained, the suggested areas for research and 
development would have been aggregated into a 
Round Three instrument and mailed to participants.) 



Round Three 

The third round of the modified Delphi study 
focused on the major topics and suggested areas of 
research and development retained from Rounds 
One and Two. In this round, participants examined 
the six major topics and their corresponding areas 
for research and development and completed two 
parts (Part A and Part B) for Round Three. The two 
parts for Round Three differed somewhat from the 
Delphi methodology of gaining consensus, imple- 
mented in Rounds One and Two. In the third round, 
participants prioritized the six major topics and 
allocated a hypothetical budget. Because of the 
uniqueness of these two parts, the authors pilot 
tested the instruments with the members of AEL's 
Rural Coordinating Unit. Results from the pilot test 
led to changes in the instrumentation. These changes 
were incorporated into the final instruments before 
they were mailed to participants. (See Appendix C, 
Round Three Instrument Documentation.) 

Part A. The authors mailed Part A of Round 
Three in early November 1992. The participants 
ranked the six major topics (school effectiveness, 
curriculum provisions, school-community partner- 
ships, human resource, use of technology, and fi- 
nance and governance issues) to reflect th? order of 
priority for each research topic. Participants ranked 
these topics from 1 to 6, with 1 being the highest 
priority and 6 being the lowest. Completed instru- 
ments were returned by mid-November 1992. 

Once all completed instruments were received, 
the authors converted the ranks to point values for 
each major topic. Before the point values were 
calculated, the authors reversed the scale so that 1 
equaled the lowest ranking and 6 equaled the high- 
est ranking. Based on this reversal, the total point 
values to allocate for all six major topics equaled 
609 points. (The total point value was calculated by 
assigning each ranking a point value: six points for 
ranking 1; five points for ranking 2; four points for 
ranking 3; three points for ranking 4; two points for 
ranking 5; and one point for ranking 6. Each topic 
had the potential to be ranked by 29 participants. 



5 15 



Appalachia Educational Laboratory • Occasional Paper 35 



Therefore, the point values were multiplied by 29 to 
receive the total point value.) After the authors cal- 
culated the point value for each major topic, the 
overall final rank order was established. 

Part B. The authors mailed participants Part B 
of Round Three in late November 1992. Partici- 
pants allocated a hypothetical research and devel- 
opment budget to address the six major topics and 
their associated areas for research and development. 
Their completed budget allocations were due to the 
authors in early December 1992. 

Part B focused on the final task for participants 
to complete. The instrumentation for Part B in- 
cluded a response card and 10 blue stick-on dots. 
The participants were to assume the role of a direc- 
tor of a newly formed national R&D center for 
rural educational research and development. In 
this role, they would have a $5,000,0002 budget to 
allocate for rural educational research and develop- 
ment. 

Participants were instructed to think of each 
blue dot as 10 percent — $500,000 — of their research 
and development budget and to allocate this bud- 
get across the six major topics (school effectiveness, 
curriculum provisions, school-community partner- 
ships, human resources, use of technology, and fi- 
nance and governance issues). Participants were 
instructed that the dot amount could not be broken 
down into smaller amounts. Participants had the 
option of using al! 10 dots or some combination of 
dots across the topics, Dots did not have to be 
allocated for each topic. Participants also had the 
option of allocating money toward "other" research 
topics. 

Data Analysis 

Findings for Rounds One and Two described 
frequencies of agreement with the importance of 
rural education research and development topics. 



Findings for Round Three described the rank order 
of topics and theoretical budget allocations. A post 
hoc analysis of Round Three results was also con- 
ducted to explore possible relationships that might . 
exist between the rank priorities and dot alloca- 
tions assigned to each topic. 

These relationships were explored by comput- 
ing Spearman rank order correlations between pri- 
ority rankings and dot allocation rankings for each 
research and development topic used in the study, 
not just those retained after Rounds One and Two. 
Next, the authors conducted a post hoc analysis of 
Round One ratings, Round Two ratings, and Round 
Three rankings to explore possible predictive rela- 
tionships between importance ratings of topics from 
Rounds One and Two and priority rankings from 
Round Three, again using all the data generated in 
Rounds One, Two, and Three. Stepwise regression 
analysis was applied to the Round One and Round 
Two rating agreement scores (independent vari- 
ables) and the Round Three (dependent variable) 
topic importance scores (rankings and budget allo- 
cations) to describe possible linear associations be- 
tween the ratings in each round of the modified 
Delphi study. 

Since the authors were conducting an entirely 
empirical post hoc analysis of the rating data col- 
lected in the modified Delphi study, no attempt 
was made to test hypotheses or to interpret the 
statistical significance of possible linear associa- 
tions between ratings from Rounds One, Two, and 
Three. Instead, the authors attempted an explora- 
tion of the data to describe potential relationships 
that might be used to generate hypotheses for fu- 
ture empirical testing with well-defined samples 
and more reliable and valid instruments designed 
to elicit specific responses from the population of 
rural education researchers. The reader is encour- 
aged to examine other summary data described in 
this report to generate additional hypotheses for 
future research exploration. 



2 Through a literature search, the authors located two articles focusing on the establishment of an annual budget for a 
national R&D center. In 1992, Atkinson and Jackson recommended that a "robust R&D cen f er" would need at least $3 
million annually in core funding to operate. Campbell et al. (1975) recommended stable funding for R & D centers at 
about $5.6 to $7.4 million per year. The authors first averaged the Campbell figure and then averaged that with the 
Atkinson and Jackson figure and determined the amount, $5 million, for use as the hypothetical budget.) 
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Selection of Participants for 
Commentaries 

Following completion of the draft paper, a se- 
lect group of RE/SIG members were invited to 
review the paper and provide commentaries. 
Again, RE/SIG members were used to facilitate 
interaction on research issues among the mem- 
bers. Using the RE/SIG membership list, the au- 
thors selected 14 members to review the paper. In 
terms of gender balance, eight of the 14 (57%) were 
male, six (43%) were female. In terms of geo- 
graphic balance, one member was from each of the 
following areas: Alabama; California; Washington, 
DC; Maine; Maryland; Nebraska; New Hampshire; 
New Mexico; New York; Oklahoma; Pennsylva- 
nia; and West Virginia. Two were from Texas. 

In terms of employer, six were employed at 
institutes for higher education; four were employed 
at Labs/ Centers; three were employed at educa- 
tional associations; and one was employed at a 
state education agency. 



Collection of Draft Paper 
Commentaries 

Following data collection and analysis, the au- 
thors shared a draft copy of the modified Delphi 
study results with the selected group of 14 mem- 
bers of the RE/SIG of AERA. These 14 members 
volunteered to peer review the draft copy and, if 
they deemed appropriate, to provide commentaries 
reflecting on the contents and implications of the 
study. This select group of professionals provided 
feedback to the authors of the paper in various 
ways. Most provided insights and suggestions, 
either written or oral, on how to strengthen the 
actual study report. The authors considered care- 
fully their suggestions and incorporated many of 
them into this final report. Some commentators 
provided advice about continued used of the Delphi 
procedure as a strategy for promotiftgrimproved 
communications among RE/SIG members. 'Cthers 
wrote commentaries suggesting uses of the menu 
for future rural educational research. These com- 
mentaries appear as Appendix D, Commentaries, 
to the study. 



17 



Priorities for Research and Development With Rural, Small Schools 



Chapter Three: 
Findings 



The findings for the three rounds of the modi- 
fied Delphi study are presented below, under their 
designated headings. 

Round One 

Twenty-eight of the 29 RE/SIG members (97%) 
returned completed Delphi instruments for Round 
One. Participants carefully read each research and 
development area presented under the major topics 
and circled the degree to which they agreed that a 
specific area was a priority. The research and de- 
velopment areas presented in Round One were the 
areas developed by the FICE Subcommittee on 
Rural Education. A five-point scale of Strongly 
Agree, Agree, Neutral, Disagree, or Strongly Dis- 
agree was used. The authors entered data from 
these instruments into SPSS-PC+ and frequencies 
were computed. Following is a description of the 
findings for Round One. 

The Round One Delphi instrument (see Appen- 
dix A) contained six major topics with a total of 44 
research and development areas within the topics. 
Of the original 44 areas, 31 (70%) were retained 
from Round One. The authors retained the re- 
search and development areas if 90 percent of the 
respondents Strongly Agreed, Agreed, or were Neu- 
tral with/toward the item. Figure 2, Summary of 
Findings for Round One, details the proportion of 
research and development areas from the original 
FICE Agenda that were retained by RE/SIG mem- 
bers during Round One of the modified Delphi 
study rating procedure. 

Of the original eight research and development 
areas generated by the FICE Subcommittee for Ma- 



jor Topic 1, six were retained by participants, for a 
retention rate of 75 percent. The research and devel- 
opment areas retained under the other five topics 
were: Major Topic 2, 20 percent (1 of 5); Major 
Topic 3, 89 percent (8 of 9); Major Topic 4, 86 per- 
cent (6 of 7); Major Topic 5, 75 percent (6 of 8); and 
Major Topic 6, 57 percent (4 of 7). 

Specific findings for each major topic are pre- 
sented below. 

Major Topic 1: The Overall Effectiveness of Rural 
Schools 

In order of decreasing agreement, participants 
held that the following areas were important re- 
search priorities in better understanding the overall 
effectiveness of rural schools: 

100% Improve access to educational opportunity 
in isolated rural communities. 

100% Assess the impact of educational reform on 
rural schools. 

96% Identify the problems unique to the delivery 
of education in isolated rural communities 
in the following special populations: handi- 
capped, disadvantaged, and gifted. 

96% Identify characteristics of effective rural 
schools. 

96% Assess the federal role in rural education. 

93% Conduct evaluation studies of student 
achievement in rural schools. 



9 

18 



Appalachia Educational Laboratory 9 Occasional Paper 35 




75% 



20% 





89% 



2LL 



Areas retained in Round One 



Original FICE Agenda Areas 



86% 




75% 




57% 



m 



1 2 3 4 5 6 

Overall Curricular School /Comm. Human Use of Financial 

Effectiveness Provisions Partnerships Resources Technology Support 

Major Topics 

Figure 2 

Summary of Findings for Round One 

10 



19 



Priorities for Research and Development With Rural, Small Schools 



Major Topic 2: Curricular Provisions in Rural 
Schools 

Participants agreed that the following area was 
an important research priority in better understand- 
ing the curricular provisions in rural schools: 

96% Provide adult literacy improvement in iso- 
lated rural communities. 

Major Topic 3: School and Community Partner- 
ships on Behalf of Rural Schools 

In order of decreasing agreement/ participants 
held that the following areas were important re- 
search priorities in better understanding school-com- 
munity partnerships on behalf of rural schools: 

100% Examine the factors of rural community 
economies that influence rural students' de- 
cisions to remain in school and graduate. 

100% Identify the social and cultural issues of iso- 
lated rural communities that impact rural 
education. 

96% Assess how federal-state-local policies are 
impacting rural schools and rural communi- 
ties. 

96% Assess the role of the rural school in promot- 
ing employability. 

93% Identify effective school/ community /private 
sector partnerships. 

93% Review legal procedures and issues pertain- 
ing to school and community partnerships 
on behalf of rural schools. 

93% Assess if Native American communities, or 
their leamingenvironments, differ from other 
rural communities. 

93% Identify effective alternative schooling pro- 
grams in the rural communities. 



Major Topic 4: Human Resources for Rural Schools 

In order of decreasing agreement participants 
held that the following areas were important re- 
search priorities in better understanding human re-« 
sources for rural schools: 

100% Identify strategies that have been successful 
for releasing rural teachers from their class- 
rooms for professional development. 

96% Identify successful strategies for the recruit- 
ment of qualified personnel to rural schools. 

96% Identify successful strategies for the reten- 
tion of qualified personnel in rural schools. 

96% Identify the strategies used by administra- 
tors to comply with the state certification 
mandates. 

93% Identify successful leadership styles of effec- 
tive rural school administrators. 

93% Assess the impact of recent state certification 
mandates on teacher availability in rural 
schools. 

Major Topic 5: Use of Technology in Rural Schools 

In order of decreasing agreement the partici- 
pants held that the following areas were important 
research priorities for better understanding the use 
of technology in rural schools: 

96% Assess the impact of the advanced technol- 
ogy on rural school curriculum. 

96% Assess the implications for instructional staff 
and support personnel who are implement- 
ing advanced technology in rural school com- 
munities. 

93 % Identify rural schools that have demonstrated 
effective use of advanced interactive instruc- 
tional technology. 
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93% Assess the effects of advanced technologies 
on traditional rural values of closeness, con- 
nection, or personal relationships in learning 
interactions. 

93% Identify the staff development strategies that 
have been most successful in helping schools, 
teachers, and support personnel embrace and 
integrate advanced technologies into their 
overall rural school system. 

93% Identify rural schools that successfully have 
implemented distance education via telecom- 
munications. 

Major Topic 6: Financial Support and Governance 
for Rural Schools 

In order of decreasing agreement, participants 
held that the following areas were important re- 
search priorities in better understanding financial 
support and governance for rural schools: 

100% Identify alternatives to school consolidation 
for rural school communities. 



100% Assess how federal and state fund distribu- 
tion formulas have impacted rural schools in 
their operations and course offerings. 

100% Assess the impact on rural schools of state 
school reform policies on course quality, di- 
versity of course offerings, and student out- 
comes. 

96% Analyze the politics of school finance in ru- 
ral communities. 

The authors also asked the participants to write 
in research and development areas under the origi- 
nal FICE Subcommittee Agenda major topics and to 
suggest other major topics. Data from Round One 
indicate that participants suggested 23 new research 
and development areas for the original FICE Agenda 
topics and that RE/SIG members proposed one new 
major topic (Unique Aspects of Rural Communi- 
ties) for consideration in subsequent rounds. Table 
1 presents the total number of RE/SIG members' 
suggested research and development areas for each 
of the original FICE Subcommittee Agenda topics. 



Table 1 

Total Allocations for RE/SIG Members' Suggested 
Research and Development Areas 

Original FICE Subcommittee RE/SIG Members 1 Suggested 

Agenda Topics Research and Development Areas 

Major Topic 1 (Overall School Effectiveness) 9 

Major Topic 2 (Curriculum Provisions) 4 

Major Topic 3 (School/Community Partnerships) 1 

Major Topic 4 (Human Resources) 2 

Major Topic 5 (Use of Technology) 4 

Major Topic 6 (Finance and Governance Issues) 3 

Total 23 
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The one new topic (Unique Aspects of Rural 
Communities) and the 23 newly suggested research 
and development areas for the original FICE Sub- 
committee Agenda topics were used to develop the 
instrument for the second round of. the modified 
Delphi study. 

Round Two 

Twenty-eight of the 29 RE/SIG members (97%) 
returned completed Delphi instruments for Round 
Two. As mentioned above, the second round in- 
strument contained the research and development 
areas and the major topic that participants suggested 
in Round One. For the second round, participants 
read carefully each suggested area presented under 
the major topics and circled the degree to which 
they agreed that this area was a priority. The same 
five-point scale used in Round One was used in 
Round Two. In addition, participants indicated the 
degree to which they agreed that the proposed new 
major topic (Unique Aspects of Rural Communi- 
ties) was a priority. If they Strongly Agreed, Agreed, 
or were Neutral with/ toward this topic, partici- 
pants were asked to suggest research and develop- 
ment areas within the topic. Figure 3, Summary of 
Findings for Round Two, details the proportion of 
RE/SIG members' suggested research and develop- 
ment areas from Round One that were retained in 
Round Two. 

Of the 23 research and development areas sug- 
gested by the RE/SIG members during Round One, 
16 (70%) were retained for subsequent rounds of the 
modified Delphi study. Of the nine research and 
development areas generated by the RE/SIG mem- 
bers for Major Topic 1, seven (78%) were retained 
for the final round of the Delphi study. Under 
Major Topic 2, two of the four new areas (50%) were 
retained; the new area proposed under Major Topic 
3 was retained (100%); and one of the two (50%) 
under Major Topic 4 was retained. Under Major 
Topic 5 f three of the four (75%) new research and 
development areas were retained by RE/SIG mem- 
bers, and two of the three (67%) were retained for 
Major Topic 6. 



Participants also indicated if they agreed that 
the suggested topic from Round One, Unique As- 
pects of Rural Communities, was a priority for re- 
search. Of the 28 respondents for Round Two, 83 
percent Strongly Agreed, Agreed, or.were Neutral., 
with/toward this topic. Based on the cut-off rate of 
90 percent, this topic was not carried over for Round 
Three. In addition, the 16 research and develop- 
ment areas suggested by participants under this 
topic were also removed from the final Delphi in- 
strument. 

Specific findings for each major topic retained 
are presented below. 

Major Topic 1: The Overall Effectiveness of Rural 
Schools 

In order of decreasing agreement, participants 
held that the following areas were important re- 
search priorities in better understanding the overall 
effectiveness of rural schools: 

96% Assess the role of rural schools in an "inte- 
grated services" approach to meeting com- 
munity needs. 

96% Assess SEA role in rural education. 

96% Assess teacher education institutions' role in 
rural education. 

96% Assess student expectations — view of the fu- 
ture. 

93% Assess the degree to which rural schools are 
educating students for participation in a na- 
tional economy versus a local economy. 

93% Assess the ways in which rural school cul- 
ture breaks down class distinctions or pro- 
motes increased cultural understanding. 

93% Understand the change process and extent 
to which change initiated in one part of school 
can encourage change throughout school cul- 
ture. 
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Major Topic 2: Curricular Provisions in Rural 
Schools 

In order of decreasing agreement, participants 
held that the following areas were important re- 
search priorities in better understanding the cur- 
ricular provisions in rural schools: 

96% Assess satisfaction of students, teachers, ad- 
ministrators, parents, and community lead- 
ers with current curriculum and instruction. 

93% Assess how state and federal curriculum de- 
velopment projects consider the needs of ru- 
ral schools. 

Major Topic 3: School and Community Partner- 
ships on Behalf of Rural Schools 

Participants agreed that the following area was 
an important research priority in better understand- 
ing school and community partnerships on behalf 
of rural schools: 

100% Assess level of parental and community in- 
volvement in rural area. 

Major Topic 4: Human Resources for Rural Schools 

Participants agreed that the following area was 
an important research priority in better understand- 
ing human resources for rural schools: 

100% Identify effective beginning teacher 
mentoring prc 0 rams for rural schools. 

Major Topic 5: Use of Technology in Rural Schools 

In order of decreasing agreement, participants 
held that the following areas were important re- 
search priorities in better understanding use of tech- 
nology in rural schools: 



96% Identify innovative, low-cost alternative pro- 
grams to those delivered via telecommuni- 
cations. 

93% Conduct technology cost-effectiveness stud- 
ies. 

93% Assess level of private support for use of 
technology in rural schools. 

Major Topic 6: Financial Support and Governance 
for Rural Schools 

Participants held that the following areas were 
important research priorities in better understand- 
ing financial support and governance for rural 
schools: 

96% Look at ways to equalize salary levels for 
teachers/administrators in rural schools com- 
pared to salary levels for those in large com- 
munities. 

96% Compare and contrast the roles and strate- 
gies of rural interest groups in the several 
school finance court cases in the states. 

Rural Education Research 
and Development Menu 

Research and development areas retained in 
Round Two were combined with the retained re- 
search and development areas from Round One to 
form the Rural Education Research and Develop- 
ment Menu used for Round Three of this modified 
Delphi study. This menu consists of the six major 
topics generated by the FICE Subcommittee on Ru- 
ral Education and 47 research and development 
areas within those topics. The research and devel- 
opment areas were generated by both the FICE 
Subcommittee and the RE/SIG members. Table 2 
details the Rural Education Research and Develop- 
ment Menu. 
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Table 2 

The Rural Education Research and Development Menu 



Major Topic 1 : The Overall Effectiveness of 
Rural Schools 

*Area A: Improve access to educational opportunity 
in isolated rural communities. 

*Area B: Identify the problems unique to the 
delivery of education in isolated rural 
communities in the following special 
populations: handicapped, disadvan- 
taged, and gifted. 

*Area C: Identify characteristics of effective ru al 
schools. 

*Area D: Conduct evaluation studies of student 
achievement in rural schools. 

*Area E: Assess the federal role in rural education. 

*Area F: Assess the impact of educational reform 
on rural schools. 

Area G: Assess the degree to which rural schools 
are educating students for participation in 
a national economy vs. a local economy. 

Area H: Assess SEA role in rural education. 

Area I: Assess teacher education institutions' role 
in rural education. 

Area J: Assess student expectations— view of the 
future. 

Area K: Assess the ways in which rural school 
culture breaks down class distinctions or 
promotes increased cultural understand- 
ing. 

Area L Assess the role of rural schools in an 

"integrated services" approach to meeting 
community needs. 

Area M: Understand the change process and 
extent to which change initiated in one 
part of school can encourage change 
throughout school culture. 



Major Topic 2: Curricular Provisions in Rural 
Schools 

*Area A: Provide adult literacy improvement in 
isolated rural communities. 

Area B: Assess satisfaction of students, teachers, 
administrators, parents, and community 
leaders with current curriculum and 
instruction. 

Area C: Assess how state and federal curriculum 
development projects consider the needs 
of rural schools. 

Major Topic 3: School and Community 
Partnerships on Behalf of Rural Schools 

*Area A: Identify effective school/community/private 
sector partnerships. 

*Area B: Assess how federal/state/local pe-;cies are 
impacting rural schools and rural communi- 
ties. 

*Area C: Review legal procedures and issues 
pertaining to school and community 
partnerships on behalf of rural schools. 

*Area D: Examine the factors of rural community 
economies that influence rural students' 
decisions to remain in school and gradu- 
ate. . 

*Area E: Assess if Native American communities, or 
their learning environments, differ from 
other rural communities. 

*Area F: Assess the role of the rural school in 
promoting employabiiity. 

*Area G: Identify effective alternative schooling 
programs in the rural communities. 

*Area H: Identify the social and cultural issues of 

isolated rural communities that impact rural 
education. 



Area I: Assess level of parental and community 
involvement in rural area. 
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Table 2 
(continued) 



Major Topic 4: Human Resources for Rural 
Schools 

*Area A: Identify successful strategies for the 

recruitment of qualified personnel to rural 
schools. 

*Area B: Identify successful strategies for the 
retention of qualified personnel in rural 
schools. 

*Area C: Identify strategies that have been success- 
ful for releasing rural teachers from their 
classrooms for professional development. 

*Area D: Identify successful leadership styles of 
effective rural school administrators. 

*Area E; Identify the strategies used by administra- 
tors to comply with the state certification 
mandates. 

*Area F: Assess the impact of recent state certifica- 
tion mandates on teacher availability in 
rural schools. 

Area G: Identify effective beginning teacher 
mentoring programs for rural schools. 

Major Topic 5: Use of Technology in Rural 
Schools 

*Area A: Identify rural schools that have demon- 
strated effective use of advanced interac- 
tive instructional technology. 

*Area B: Assess the impact of the advanced technol- 
ogy on rural school curriculum. 

*Area C: Assess the effects of advanced technolo- 
gies on traditional rural values of close- 
ness, connection, or personal relationships 
in learning interactions. 

*Area D: Assess the implications for instructional 
staff and support personnel who are 
implementing advanced technology in rural 
school communities. 



*Area E: Identify the staff development strategies 
that have been most successful in helping 
schools, teachers, and support personnel 
embrace and integrate advanced technolo- 
gies into their overall rural school system. 

*Area F: Identify rural schools that successfully 

have implemented distance education via 
telecommunications. 

Area G: Conduct technology cost-effectiveness 
studies. 

Area H: Assess level of private support for use of 
technology in rural schools. 

Area I: Identify innovative, low-cost alternative 
programs to those delivered via telecom- 
munications. 

Major Topic 6: Financial Support and 
Governance for Rural Schools 

*Area A: Analyze the policies of school finance in 
rural communities. 

*Area B: Identify alternatives to school consolidation 
for rural school communities. 

*Area C; Assess how federal and state fund 

distribution formulas have impacted rural 
schools in their operations and course 
offerings. 

*Area D: Assess the impact on rural schools of state 
school reform policies on course quality, 
diversity of course offerings, and student 
outcomes. 

Area E: Look at ways to equalize salary levels for 
teacher/administrators in rural schools 
compared to salary levels for those In 
large communities. 

Area F: Compare and contrast the roles and 

strategies of rural interest groups in the 
several school finance court cases in the 
states. 



♦Original items, generated by the FICE Subcommittee, retained. 
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Table 3 details the origins of these 47 research 
and development areas and indicates under which 
of the six major topics the areas belong. 

Round Three 

The third round of the Delphi study consisted 
of two parts. Using the Rural Education Research 
and Development Menu (above) as the final Delphi 
instrument, participants: (A) prioritized the six 
major topics and (B) developed a research and 
development budget to address the six major top- 
ics. Findings for each part will be discussed below. 



Part A. In Part A, participants reviewed the six 
major topics and their associated research and de- 
velopment areas. Next, participants ranked the six 
major topics from 1 to 6, with 1 being the highest 
and 6 the lowest, to reflect the order of priority in 
which they believed the topics should be researched. 

All 29 participants returned completed rank- 
ing forms. The authors entered data from these 
forms into SPSS-PC+ and descriptive statistics were 
computed. Table 4 displays the six major topics 
and the frequencies of the rankings per each topic 
by the 29 participants. 

The authors converted findings from Table 4 to 



Table 3 

Origins of the Research and Development Areas 
on the Rural Education Research and Development Menu 



Original FICE 
Subcommittee 
Agenda Topics 

Major Topic 1 
(Overall School 
Effectiveness) 

Major Topic 2 

(Curriculum 

Provisions) 

Major Topic 3 

(School/Community 

Partnerships) 

Major Topic 4 
(Human Resources) 

Major Topic 5 
(Use of Technology) 



FiCE 
Generated 
R&D Areas 

6 (46%) 



1 (33%) 



8 (89%) 

6 (86%) 
6 (67%) 



RE/SIG Member 
Generated 
R&D Areas 

7 (54%) 



(67%) 



1 (11%) 



1 

3 



(14%) 
(33%) 



Total 
R&D Areas 
Per Topics 

13 



7 
9 



Major Topic 6 

(Finance and Governance 

Issues) 



2 (33%) 



Totals 



31 



16 



47 
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point values for each of the six major topics. These rank order for each topic, the point value for each of 
point values were then used to determine the rank the six major topics, and the percentage of the total 
order of the six major topics. Table 5 displays the point value for each of the topics. 



Table 4 

Frequencies of Rankings Per Each of the Major Topics 
Agenda Topics RE/SIG Members' Rankings 



1 (highest) 2 



Major Topic 1 
(Overall School 
Effectiveness) 

Major Topic 2 
(Curriculum Provisions) 

Major Topic 3 

(School/Community 

Partnerships) 

Major Topic 4 
(Human Resources) 

Major Topic 5 
(Use if Technology) 

Major Topic 6 
(Finance and 
Governance Issues) 



Ranking 
1 
2 

3 
4 
5 



17 
1 

6 
2 
1 



4 

2 

6 
6 
4 



1 
5 

4 
7 
8 



4 

5 

5 
7 

5 



2 
6 

3 
6 



6 (lowest) 
1 

10 

5 
1 

5 



Table 5 

Point Values for Major Topics 



Major Topics 



1 



(Overall School Effectiveness) 
3 

(School/Community/ 
Partnerships) 

4 

(Human Resources) 
6 

(Finance and Governance Issues) 
5 

(Use of Technology) 
2 

(Curriculum Provisions) 
19 



Point Value 
Per Topic 

143 
108 

104 
91 
90 

73 



Percentage of 
Total Point Value 

23% 



18% 

17% 
15% 
15% 

12% 
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The RE/SIG members seemed to have a clear 
idea of which of these topics they felt ranked high- 
est and lowest in terms of priority for research and 
development; however, they were less clear in terms 
of the middle-scale rankings. Table 5 indicates that 
70 points separated the topic ranked highest in pri- 
ority from the topic ranked lowest. However, only 
18 points separated the topic ranked second in pri- 
ority from the topic ranked next to last in order of 
priority. 

The RE/SIG members ranked Major Topic 1: 
The Overall Effectiveness of Rural Schools as hav- 
ing the highest priority for research and develop- 
ment. A 35-point difference separated this topic 
from the topic ranked number 2, Major Topic 3: 
School and Community Partnerships on Behalf of 
Rural Schools. Only four points separated Major 
Topic 3 from Major Topic 4: Human Resources for 
Rural Schools, which was ranked third in terms of 
priority for research and development. Major Topic 
6: Financial Support and Governance for Rural 
Schools was ranked fourth in order of priority, 13 



points below the topic ranked third. Just one point 
separated the fourth and fifth ranking. Major Topic 
5: Use of Technology in Rural Schools received 90 
points and ranked fifth. Major Topic 2: Curricular 
Provisions in Rural Schools was viewed as least im- 
portant in terms of priority for research and develop- 
ment in rural education. This topic ranked 17 points 
below the topic ranked fifth. 

Part B. In Part B, the RE/SIG members assumed 
that they were a director of a newly-formed national 
R&D center for rural educational research and de- 
velopment. In this role, they were given a hypotheti- 
cal $5,000,000 budget to allocate for rural educa- 
tional research and development for FY 93. 

All 29 RE/SIG members returned completed re- 
sponse cards for Part B. Data for total number of 
dots per major topic were entered into SPSS-PC+ 
and descriptive statistics were computed. The find- 
ings for Part B are explained in the following two 
tables. Table 6 displays the frequencies of dots per 
major topic allocated by the 29 participants. 



Table 6 

Frequencies of Dot Allocations Per Major Topic 
Agenda Topics Frequencies of Dots ($500,000 each) 

0 123456789 10 

Major Topic 1 
(Overall School 

Effectiveness) 1 10 7 6 4 1 

Major Topic 2 
(Curriculum 

Provisions) 7 15 5 1 1 

Major Topic 3 
(School/Community 

Partnerships) 2 14 9 4 

Major Topic 4 

(Human Resources) 3 11 14 1 

Major Topic 5 

(Use of Technology) 1 10 13 5 

Major Topic 6 
(Finance and 

Governance Issues) 1 13 9 6 
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This table indicates the wide array of allocation 
utilized by the RE/SIG members. For example, for 
Major Topic 1, 10 participants allocated one dot; 
seven allocated two dots; six allocated three dots; 
four allocated four dots; one allocated five dots; 
and one did not allocate any funds for this topic. 



Of interest on this table is that seven participants 
did not allocate any funds toward Major Topic 2. 

Table 7 displays the mean allocation assigned 
by participants to each major topic. The major 
topics are presented in descending order, in terms 
of their mean number of dots allocated and their 
mean allocation. 



Table 7 

Mean Allocation Allotted by Participants Per Major Topic 



Major Topics 



1 



(Overall School Effectiveness) 



(Use of Techology) 
6 

(Finance and 
Governance Issues) 



(School/Comm unity 
Partnerships) 



(Human Resources) 



(Curriculum 
Provisions) 

Total 



Mean Number of Dots 
Allocated 

2.172 



1.759 



1.690 



1.517 



1.483 



1.207 



Mean 
Allocation 

$1,086,000 
$879,500 
$845,000 

$758,500 

$741,500 
$603,500 

$4,914,000 



Major Topic 1 received the highest mean allo- 
cation ($1/086,000) with Major Topic 2 receiving 
the lowest mean allocation ($603,500), The differ- 
ence in mean allocations ranged from a high of 
$206,500 (difference between Major Topic 1 and 
Major Topic 5, ranked first and second in terms of 
funding) to a low of $17,000 (difference between 
Major Topic 3 and Major Topic 4, ranked fourth 
and fifth in terms of funding). Major Topics 5 and 
6 ranked second and third, respectively, in mean 
allocations, differed by $34,500. Participants 



seemed to have more difficulty differentiating be- 
tween Major Topics 5 and 6 and Major Topics 3 and 
4. In terms of allocating funding, participants gave 
similar funding to Topics 5 and 6 and Topics 3 and 
4. 

Table 8 compares the results from Part A and 
Part B of Round Three. This table displays the 
rankings for all six major topics in terms of their 
priority for research and development and in terms 
of their mean allocation for research and develop- 
ment funding. 
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Table 8 

Comparison ot > likings for Part A and Part B of Round Three 



Rank for Mean 

Major Topics Priority Rank R&D Allocation of Funding 



1 1 1 

(Overall School 
Effectiveness) 

2 6 6 
(Curriculum 

Provisions) 

3 2 4 
(School/Community 

Partnerships) 

4 3 5 
(Human Resources) 

5 5 2 
(Use of Technology) 

6 4 3 
(Finance and 

Governance Issues) 



Major Topics 1 and 2 were ranked consistently 
by all 29 participants in both Parts A and B of 
Round Three. In other words, participants viewed 
Major Topic 1 as having the highest research and 
development priority and, therefore, they allocated 
the most research and development dollars toward 
this topic, On the other hand, Major Topic 2 was 
viewed as least important in terms of priority and 
the allocated budget reflects these views. 

The other four topics were not ranked consis- 
tently across both parts; however, their rankings 
are interesting, In terms of priority for research 
and development, Major Topics 3 and 4 were ranked 
second and third; but in terms of mean allocation of 
funding, these two topics were ranked fourth and 
fifth. Similar rankings occurred with Major Topics 
5 and 6. These two topics were ranked fifth and 
fourth, respectively, in terms of priority for research 



and development; but they were ranked second 
and third in terms of budget allocation. 

Post-hoc analysis. After the authors completed 
an initial descriptive statistical analysis of the Round 
Three ranking and dot allocation data, they con- 
ducted an exploratory post hoc analysis of potential 
relationships between the rank assignments and 
funding allocations, The purpose of the post hoc 
analysis was to describe any statistically significant 
findings that could be interpreted to suggest pos- 
sible variable relationships. Knowledge of these re- 
lationships could be used to simplify future sur- 
veys related to the rural education research topics 
or to suggest hypotheses for future studies. 

The reader is reminded that the authors con- 
ducted an entirely empirical post hoc analysis of 
the rating data collected in the modified Delphi 
study. The authors had no hypotheses to test or 
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any substantive theory to guide interpretation of 
the statistical significance of possible linear associa- 
tions between ratings from Rounds One, Two, and 
Three. Moreover, the sample of opinions from which 
these data were generated was not randomly se- 
lected from a well-defined population of rural edu- 
cation researchers. As a result, the distributions of 
the sample data do not meet basic assumptions for 
use of the common statistical tests described below. 
Therefore, no statistically valid interpretation of the 
following results was attempted. None should be 
inferred by the reader. 

Instead, this exploration of the data was de- 
signed to use standard statistical significance test- 
ing thought to be appropriate for this type of data 
to describe potential hypothetical relationships 
among the modified Delphi ratings from three 
rounds of data generation. The only relationships 
discussed below were those that were found to be 
statistically significant in this limited set of nonran- 
dom data. 

In the case of the Spearman rank correlation 
statistic, a correlation value and approximate statis- 
tical significance level are offered for the reader's 
information. In the case of the stepwise regression 
results, only the metric regression coefficient is pre- 
sented to suggest a possible direction for the hypo- 
thetical relatior " .ips described. No additional sta- 
tistical information is given to avoid over interpre- 
tation of these questionable statistical tests by read- 
ers. The reader is encouraged to examine this evi- 
dence with skepticism and to use other summary 
data described in this report to generate additional 
hypotheses for future research exploration. 

Possible association of rank assignments and 
funding allocations. The authors found three of 
the 36 (8%) possible Spearman rank correlations 
between topic ranks and funding allocations to be 
statistically significant (p < 0.10 with 19 degrees of 
freedom). First, as the rank assigned to Major Topic 
1 (Overall Effectiveness of Rural Schools) rose, the 
amount of funding allocated to the topic appeared 
to decrease (r = -0.4846, p < 0.035). Second, as the 
rank i signed to Major Topic 3 (School and Com- 
munity Partnerships) rose, the amount of funding 
allocated to Major Topic 1 appeared to increase 
(r = 0.64, p < 0.003). Finally, as the rank assigned to 



Major Topic 3 rose, the amount of funding allo- 
cated to Major Topic 3 decreased (r = -0.42, 
p < 0.08). 

These findings could be interpreted to suggest - 
that, when individuals are asked to rank prioritize 
these rural education topics, and then to allocate 
research funding amounts differentially to the same 
topics, they logically relate some topics with others 
while treating the rest as independent topics for 
research exploration. 

What seemed to be mildly surprising was the 
fact that only three of the possible 36 topic relation- 
ships were found to be statistically significant, since 
the wording of the topics suggests more correlated 
topics than the two (Topics 1 and 3) empirically 
verified through the ranking and dot allocation 
processes. The low sample size in this study and 
the moderately reliable modified Delphi process 
rating system may help to account for the low num- 
ber of statistically significant associations described. 
In the future, researchers may wish to use similar 
correlational analysis research to more systemati- 
cally investigate relationships between these rural 
education research topics. 

Possible prediction of rankings and funding 
allocations from Round One and Round Two rat- 
ings of agreement with research and development 
areas. Possible relationships (linear regression co- 
efficients) between Round One and Round Two 
ratings (independent variables for each topic) and 
Round Three rankings and funding allocations (de- 
pendent variables for each topic) were explored via 
stepwise regression analysis (p < .10 tolerance lim- 
its for tests of regression coefficient significance). 
One regression analysis was conducted for each of 
the six major topics considered. The purpose of 
this post hoc analysis was to describe whether 
rankings and funding allocation outcomes could 
be predicted from knowledge of previous "agree- 
ment" ratings of the same issues. The results are 
summarized below. 

• Major Topic 1: The Overall Effectiveness of 
Rural Schools 

The rating of Area G (Conduct evaluation 
studies of student achievement in rural schools) 
[fe = 1.39] from Round One and the rating of Area B 
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Assess the students who leave the rural community 
and become successful either through advanced edu- 
cation or professional life [fe = 1.03] from Round 
Two could be used to predict the priority ranking 
assigned to Major Topic 1 in Round Three. Basi- 
cally, as disagreement with these area statements G 
and B increased, rank scores increased. However, 
none of the area ratings from Rounds One and Two 
could be used to predict the funding allocations 
assigned to Major Topic 1 in Round Three. 

• Major Topic 2: Curricular Provisions in Rural 
Schools 

Only the rating of Area D (Improve coopera- 
tion with the private sector for rural education) 
[b = -0.88] from Round One could be used to predict 
the priority ranking assigned to Major Topic 2 in 
Round Three. As agreement with this area in- 
creased, the ranking score for Major Topic 2 in- 
creased. None of the area ratings from Rounds One 
and Two could be used to predict the funding allo- 
cations assigned to Major Topic 2 in Round Three. 

• Major Topic 3: School and Community Part 
nerships on Behalf of Rural Schools 

Only the rating of Area ? (Identify effective 
postsecondary and adult education programs in 
rural communities) [b = -1.53] from Round One 
could be used to predict the priority ranking as- 
signed to Major Topic 3 in Round Three. As agree- 
ment with Area F increased, the rank scores for 
Major Topic 3 increased. None of the area ratings 
from Rounds One and Two could be used to predict 
the funding allocations assigned to Major Topic 3 in 
Round Three. 

• Major Topic 4: Human Resources for Rural 
Schools 

Only the rating of Area G (Assess the impact of 
recent state certification mandates on teacher avail- 
ability in rural schools) (b = -0.96) from Round One 
could be used to predict the priority ranking as- 
signed to Major Topic 4 in Round Three. As agree- 



ment with Area G increased, the rank scores for 
Major Topic 4 increased. None of the area ratings 
from Rounds One and Two could be used to predict 
the funding allocations assigned to Major Topic 4 in 
Round Three. 

• Major Topic 5: Use of Technology in Rural 
Schools 

None of the area ratings from Rounds One and 
Two could be used to predict either the topic prior- 
ity ranking or funding allocations assigned to Ma- 
jor Topic 5 in Round Three. 

• Major Topic 6: Financial Support and 
Governance for Rural Schools 

Only the rating of Area B (Analyze the politics 
of school finance in rural communities) [b = 1.05] 
from Round One could be used to predict the prior- 
ity ranking assigned to Major Topic 6 in Round 
Three. As disagreement with Area B increased, the 
ranking score for Major Topic 6 increased. None of 
the area ratings from Rounds One and Two could 
be used to predict the funding allocations assigned 
to Major Topic 6 in Round Three. 

It appears as if a few areas of agreement from 
previous measures of respondent ratings of these 
rural education topics can be described in a direct 
linear relationship to later priority rankings or fund- 
ing allocations of the same topics. No relationship 
could be drawn between agreement ratings and 
funding allocations. If these findings are not en- 
tirely due to chance, future research may simplify 
estimation of ranking and funding allocation out- 
comes by using simple agreement rating items to 
"predict" these outcomes. More importantly, re- 
searchers may gain additional focus for future stud- 
ies on these topics by exploring the conceptual rela- 
tionships between the research and development 
areas represented in the rating scales of Rounds 
One and Two, and the six major rural education 
research topics that this study has empirically veri- 
fied as worthy of future study. 
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Chapter Four: 
Conclusions 



The authors drew certain conclusions from the 
findings presented above. These conclusions are 
presented below and are summarized under three 
themes: Uses of the Rural Education Research and 
Development Menu, Implications for Future Rural 
Educational Research and Development, and Im- 
plications for Future Delphi Studies. 

Uses of the Rural 
Education Research and 
Development Menu 

For readers, the authors and commentators sug- 
gest the following uses of the Rural Education Re- 
search and Development Menu described in this 
report. 

• Review topics and areas on the menu. Develop 
a process for involving rural educators, students, 
and citizens (regional, statewide, or local) to 
determine priorities for the topics and areas on 
the menu. 

• Develop specific empirical questions for priority 
topics and areas. 

• Identify other research teams that are interested 
in working on these issues. Form research 
collaborations on rural education issues. 

• Create a dialogue among educational 
researchers that focuses on high priority rural 
education topics and areas. Consider using 
Internet (or other networks) to facilitate this 
dialogue. 



• Evaluate periodically the validity of the Rural 
Education Research and Development Menu's 
topics and areas. 

Implications for Future 
Rural Educational Research 
and Development 

Certain implications for future rural educational 
research and development can be derived from this 
study. These implications are as follows: 

• The RE/SIG members validated the contents of 
the original 1991 FICE Agenda topics and 
research and development areas. Through two 
rounds of consensus building, these RE/SIG 
members indicated that they agreed with the 
content validity of the original FICE Agenda's 
six major topics. In addition, the RE/SIG 
members agreed with the content validity of 31 
of the 44 (70%) original FICE Agenda research 
and development areas within each of the six 
topics. 

• The RE/SIG members expanded the contents 
of the original 1991 FICE Agenda research and 
development areas. RE/SIG members 
suggested 23 new research and development 
areas, across the six major topics, during the 
first round of this study. In Round Two, the 
RE/SIG members validated the contents of these 
new areas and agreed on 16 of them for inclusion 
in Round Three. Therefore, the RE/SIG 
members expanded the original FICE Agenda 
by 16 research and development areas. 
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• This research effort produced an updated 
menu of researchable issues available for use 
by rural educational researchers. Building 
upon the earlier efforts of the FICE 
Subcommittee on Rural Education, the RE/ 
SIG members added their knowledge and 
insights and adapted the original 1991 Agenda 
to reflect changes in the conditions of rural 
educational research. The Rural Education 
Research and Development Menu should 
continue to facilitate a dialogue on the 
problems and contributions of rural education, 
encourage research and development on rural 
concerns, and promote coordination and 
collaboration among educational researchers. 

♦ In addition to an updated agenda, this research 
effort provided a first attempt at prioritizing 
the six major topics and their research and 
development areas. This priority ranking, 
along with the hypothetical budget allocations, 
may assist rural educational researchers with 
decisions concerning their research and 
development resources. 

Implications for Future 
Delphi Studies 

As mentioned above, this study used a modi- 
fied Delphi technique in that participants did not 
gain consensus on the Round Three activities: pri- 
ority ranking and budget allocation. Certain im- 



plications can be derived from these activities for 
future Delphi studies. These implications follow: 

• The authors' modification of this Delphi 
procedure, the priority ranking and allocation 
of a hypothetical budget, could provide 
educational researchers with a starting point 
for future Delphi studies. In this study, RE/ 
SIG members had difficulty differentiating 
among four of the six major topics in terms of 
priority and budget allocation. Future Delphi 
participants could first be asked to gain 
consensus on the priority rankings before they 
work toward consensus on the budget 
allocations. Agreeing on the priority rankings 
before allocating a hypothetical budget might 
help alleviate some of the difficulties 
experienced by the RE/ SIG members in this 
modified Delphi study. 

• An attempt to describe possible post hoc 
relationships between major topic ranks and 
funding allocations did little to clarify the 
findings of this study. Only a few of the 
expected relationships between topics were 
empirically supported by evidence measured 
in this study. Another suggestion for future 
Delphi studies might be to systematically study 
the rating relationships among topics through 
factor analysis. Use of a reduced set of topic 
factors may enable potential raters to more 
precisely identify and prioritize the items. 
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Improving Education Through 
Reaearch and Development 



MEMORANDUM 



TO: Rural Education SI6 Members 

FROM: John R. Sanders 

DATE: February 6, 1992 

SUBJECT: Delphi Study 



Thank you for agreeing to participate in our RE/SIC Delphi study* The 
purpose of the study Is to identify a consensus list of priority research 
topics for our SIG. 

This is the first of three questionnaires we will use in this study* Ve 
hope you will hang in there with us for these several rounds of questions- 
At the end, we will assess whether or not the effort has been worthwhile 
and whether there are viable collaborative inquiries that we should 
consider — perhaps with other SIGs or with other organisations* 

But first things first. Please complete the enclosed questionnaire and 
mail It back in the return envelope ASAP* Ve will compile the results 
and get them back to you with a second round questionnaire* Thanks again 
for participating. 



JRS:ksc 
Enclosures 



Appalachia Educational Laboratory • A nonprofit corporation * An equfl/ opportunity /affirmative action employer 
P.O. Box 1348. Charleston, WV 25325-1348 • Shipping: 1031 Quarter Street, Charleston. WV 25301-2314 
Telephone 304/347-0400 <■ Toll free 800/624*9120 • FAX 304/347-0487 
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Areas for Research and Development on Rural Education 



In 1989, the Federal Interagency Committee on Education (FICE) 
Subcommittee on Rural Education examined the state of rural education 
research, drawing upon the knowledge of specialists within the federal 
government. From this examination, six topics emerged as priorities for 
future research on rural education. These six priority topics, which are 
described In An Agenda for Research and Development on Rural Education , 
are: The Overall Effectiveness of Rural Schools; Curricular Provisions 
In Rural Schools; School and Community Partnerships on Behalf of Rural 
Schools; Human Resources for Rural Schools; Use of Technology in Rural 
Schools; and Financial Support and Governance for Rural Schools. For each 
topic, members of the Subcommittee generated sample research questions or 
areas of focus. 

Below are the six priority topics and the research questions as 
generated by the FICE Subcommittee on Rural Education. As a participant 
In Round One of this Delphi study, please read carefully each area 
presented under the six major topics and circle the degree to which you 
agree that this area is a priority: SA (Strongly Agree), A (Agree), 
N (Neutral), D (Disagree), and SD (Strongly Disagree). 

Following the FICE Subcommittee items is an opportunity for you to 
nominate other areas and topics, not included on the Subcommittee's 
agenda* Please complete the demographic Items on the last page of the 
questionnaire. 



Major Topic 1: The Overall 
Effectiveness of Rural Schools 

(Researchers should clearly define the factors that describe and affect 
the rural community being studied, such as geographic isolation; economy 
of scale; and variability in culture, economy, and social environment.) 

Area A. Identify successful instructional delivery practices In rural 
schools that can be replicated. 

SA A N D SD 

Area B. Improve access to educational opportunity In isolated rural 
communities. 

SA A N D SD 

Area C. Improve school dropout prevention programs in rural schools. 
SA A N D SD 
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Area D. Identify the problems unique to the delivery of education in 

isolated rural communities in the following special populations: 

Handicapped SA A H D SD 

Disadvantaged SA A N D SD 

Gifted SA A N D SD 

Area E. Identify characteristics of effective rural schools. 

SA A N D SD 

Area F. Assess the Impact of educational reform on rural schools. 

SA A N D SD 

Area G* Conduct evaluation studies of student achievement In rural 
schools* 

SA A N D SD 

Area H. Assess the federal role in rural education. 

SA A N D SD 

Area I. Within this topic, Overall Effectiveness of Rural Schools, what 
other areas of research, not listed above, should be examined? 
(Please begin areas vith a verb 0 as preceding examples.) 



Major Topic 2: Curricular 
Provisions in Rural Schools 

(Serious curricular concerns have been raised over needs assessment , 
individualised instruction, design and Implementation, cooperation with 
private sector development, access, and adult literacy improvement*) 

Area A. Identify effective needs assessment techniques for rural 
education* 

SA A N D SD 

Area B* Identify individualized Instruction plans In isolated rural 
communities. 

SA A N D SD 
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Area C. Assess curriculum design and/or implementation* 
SA A N D SD 

Area D. Improve cooperation with the private sector for rural education* 
SA A N D SD 

Area E. Provide adult literacy Improvement in Isolated rural communities. 

SA A N D SD 

Area F* Within this topic, Curricula r Provisions in Rural Schools, what 
other areas of research, not listed above, should be examined? 
(Please begin areas with a verb, as preceding examples*) 



Major Topic 3: School and Community 
Partnerships on Behalf of Rural Schools 

(Research on school and community relationships should describe the 
environment within which learning occurs*) 

Area A. Identify effective school/community/private sector partnerships* 

SA A N D SD 

Area B. Assess how federal-state-local policies are impacting rural 
schools and rural communities* 

SA A N D SD 

Area C* Review legal procedures and issues pertaining to school and 
community partnerships on behalf of rural schools* 

SA A H D SD 

Area D* Examine the factors of rural community economies that Influence 
rural students 9 decisions to remain In school and graduate* 

SA A N D SD 

Area £• Assess if Native American communities, or their learning environ- 
ments, differ from other rural communities* 

SA A N D SD 
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Area P. Identify effective postsecondary and adult education programs in 
rural communities* 

SA A N D SD 

Area G. Assess the role of the rural school in promoting employability. 

SA A N D SD 

Area H. Identify effective alternative schooling programs in the rural 
communities. 

SA A N D SD 

Area I. Identify the social and cultural Issues of isolated rural 
communities that Impact rural education? 

SA A N D SD 

Area J. Within this topic, School and Community Partnerships on Behalf 
of Rural Schools, what other areas of research, not listed 
above, should be examined? (Please begin areas with a verb, as 
preceding examples.) 



Major Topic hi Human Resources 
for Rural Schools 

(Research on rural school personnel should be focused on recruitment, 
retention, professional development, administration, and supervision.) 

Area A. Identify successful strategies for the recruitment of qualified 
personnel to rural schools. 

SA A N D SD 

Area B. Identify successful strategies for the retention of qualified 
personnel in rural schools. 

SA A N D SD 

Area C. Identify strategies that have been successful for releasing rural 
teachers from their classrooms for professional development. 

SA A N D SD 
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Area D. Identify successful teaching styles of effective rural school 
educators. 

SA A N D SD 

Area E. Identify successful leadership styles of effective rural school 
administrators. 

SA A N D SD 

Area F. Identify the strategies used by administrators to comply with 
the state certification mandates. 

SA A N D SD 

Area 6. Assess the impact of recent state certification mandates on 
teacher availability In rural schools. 

SA A N D SD 

Area H. Within this topic. Human Resources for Rural Schools, what other 
areas of research, not listed above, should be examined? (Please 
begin areas with a verb, as preceding examples.) 



Major Topic 5; Use of Technology 
in Rural Schools 

(The learning outcomes achieved from each new technological advance need 
to be studied— individually and comparatively.) 

Area A. Identify rural schools that have demonstrated effective use of 
advanced Interactive Instructional technology. 

SA A N D SD 

Area B. Assess the Impact of the advanced technology on rural school 
currl culum. 

SA A N D SD 

Area C. Assess the impact of the advanced technology on rural school 
students 9 outcomes. 

SA A N D SD 
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Area D. Assess the effects of advanced technologies on traditional rural 
values of closeness, connection, or personal relationships In 
learning interactions* 

SA A N D SD 

Area E* Assess the implications for instructional staff and support 
personnel Who are implementing advanced technology in rural 
school communities* 

SA A N D SD 

Area F* Identify the staff development strategies that have been most 
successful in helping schools, teachers, and support personnel 
embrace and integrate advanced technologies into their overall 
rural school system* 

SA A N D SD 

Area G. Identify rural schools that successfully have implemented 
distance education via telecommunications* 

SA A N D SD 

Area H* Assess the computer literacy of rural school communities that 
have Implemented advanced technology* 

SA A N D SD 

Area I. Within this topic, Use of Technology in Rural Schools, what 

other areas of research, not listed above, should be examined? 
(Please begin areas with a verb, as preceding examples*) 



Major Topic 6: Financial Support 
and Governance for Rural Schools 

(Research should focus on the effects of school aid financial distribu- 
tion formulas used by the states and by the federal government, the 
impact of school consolidation, and Issues of education standards and 
quality*) 

Area A* Identify alternative funding systems* 

SA A H D SD 
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Area B. Analyze the politics of school finance in rural communities. 

SA A N D SD 

Area C. Assess the Impact of school district restructuring and 
consolidation on the rural school connnunities. 

SA A N D SD 

Area D. Identify alternatives to school consolidation for rural school 
communities. 

SA A N D SD 

Area E. Identify successful strategies to increase rural issues aware- 
ness in state governments. 

SA A N D SD 

Area F. Assess how federal and state fund distribution formulas have 

Impacted rural schools In their operations and course offerings. 

SA A N D SD 

Area G. Assess the Impact on rural schools of state school reform 

policies on course quality, diversity of course offerings, and 
student outcomes. 

SA A N D SD 

Area H. Within this topic, Financial Support and Governance for Rural 

Schools, what other areas of research, not listed above, should 
be examined? (Please begin areas with a verb, as preceding 
examples.) 



Other Major Topics 

Are there any other major topics not Included in the Subcommittee* a agenda 
that you would like to nominate for consideration In subsequent rounds of 
this study? 
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Demographic Information 



Name (optional): 



Your professional role (check one): 

Central Office Staff 

Local Board of Education 

Principal/ Assistant Principal 

Teacher 

Higher Education Administrator 

Higher Education Professor 

CSSO/Staff 

Gove rnor / S taf f 

State Board of Education 

State Legislator/Staff 

Researcher 

Other 

Your employing agency (check one): 

Assoc! ati on 

Lab/Center 

SEA 

H/HS 

LEA 

USDOE 

IHE 

Policy 

Other 

ISA 
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Improving Education Through 
Research and Development 



MEMORANDUM 



TO: 
FROM: 1 
BATE: 




fjRural Education SIG Member* 
John R. Sanders 



May 29, 1992 



Thank you for participating in Round One of the Delphi study for the 
Rural Education SIG (RE/SIG) group. Of the 28 RE/SIG membere* selected 
to participate in the study, all returned usable instruments and the data 
were aggregated for Round Two, 

Of the original 44 items in the Round One Delphi instrument, 31 were 
retained for Round Two. Items were retained if 90 percent of the respon- 
dents Strongly Agreed, Agreed, or were Neutral with? toward the item. 
(See Attachment #1 for a complete list of items retained for Round Two.) 

The second round of the Delphi study (Attachment #2) contains the items 
participants provided in Round One as write-in answers for other areas of 
research, not included on the Subcommittee 9 a agenda, that participants 
felt should he examined. 

In the second round of the Delphi study, we would like you to review 
these suggested areas of research and tell us the degree to which you 
agree that these areas are necessary for inclusion in later rounds of the 
Delphi study . Also, one participant provided a suggestion for a major 
topic, not included on the Subcommittee's agenda, that should be examined. 
We would like you, in Round Two, to tell us the degree to which you agree 
that this major topic should be added to the agenda developed by the 
subcommittee. If you circle SA, A, or N for this major topic, listed as 
Major Topic 7, please identify and list areas of research related to this 
topic that you feel should be examined more closely. 

Please return the survey in the enclosed self-addressed, stamped envelope 
by Wednesday, June 17, 1992. 

JRSikac 
Attachments 



^Originally, a purposeful sample of 30 RE/SIG members was selected for 
this study. One selected member did not return our calls and therefore 
did not participate in the study. The other RE/SIG member returned a 
completed survey to AEL; however, the survey was received well paat the 
deadline and was not included in the Round One results* 



Appslachia Educational Laboratory • 1031 Ouarrier Street • Post Office Box 1348 • Charleston. West Virginia 2S32S 
Telephone 800M2+9120 (outside West Virginia) • 800/344-6646 (in West Virginia) . 347-0400 ( in Charleston area) 
FAX (304) 347-0487 • A nonprofit corporation • An Equal Opportunity/Affirmative Action Employer 
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ATTACHMENT #1 



ITEMS RETAINED FROM ROUND ONE 



Major Topic 1: The Overall Effectiveness of Rural Schools 

- Improve access to educational opportunity In Isolated rural 
communities. 

- Identify the problems unique to the delivery of education in 
isolated rural communities in the following special populations: 
handicapped, disadvantaged, and gifted. 

- Identify characteristics of effective rural schools. 

- Assess the impact of educational reform on rural schools. 

- Conduct evaluation studies of student achievement in rural schools. 

- Assess the federal role in rural education. 



Major Topic 2: Curricular Provisions in Rural Schools 

- Provide adult literacy improvement in isolated rural communities. 



Major Topic 3; School and Community Partnerships on Behalf of Rural 
Schools 

- Identify effective school/community/private sector partnerships. 

- Assess how federal-state-local policies are impacting rural 
schools and rural communities. 

- Review legal procedures and issues pertaining to school and 
community partnerships on behalf of rural schools. 

- Examine the factors o£ rural community economies that Influence 
rural students 1 decisions to remain in school and graduate. 

- Assess if Native American communities, or their learning 
environments, differ from other rural coonunities. 

- Assess the role of the rural school in promoting employ ability. 

- Identify effective alternative schooling programs in the rural 
communities. 

- Identify the social and cultural Issues of isolated rural 
communities that impact rural education. 
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Major Topic 4; Human Resources for Rural Schools 

- Identify successful strategies for the recruitaent of qualified 
personnel to rural schools. 

- Identify successful strategies for the retention of qualified 
personnel In rural schools . 

- Identify strategies that have been successful for releasing rural 
teachers froa their classrooms for professional development. 

- Identify successful leadership styles of effective rural school 
administrators . 

- Identify the strategies used by administrators to comply with the 
state certification mandates'. 

- Assess the Impact of recent state certification mandates on 
teacher availability In rural schools. 



Major Topic 5: Use of Technology in Rural Schools 

- Identify rural schools that have demonstrated effective use of 
advanced interactive instructional technology. 

- Assess the impact of the advanced technology on rural school 
curriculum. 

- Assess the effects of advanced technologies on traditional rural 
values of closeness , connection, or personnel relationships in 
learning interactions « 

- Assess the implications for instructional staff and support 
personnel who are implementing advanced technology in rural school 
communities. 

- Identify the staff development strategies that have been most 
successful in helping schools, teachers, and support personnel 
embrace and integrate advanced technologies into their overall 
rural school system. 

- Identify rural schools that successfully have Implemented distance 
education via telecommunications. 
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Major Topic 6; Financial Support for Governance for Rural Schools 

- Analyze the policies of school finance in rural communities. 

- Identify alternatives to school consolidation for rural school 
communities* 

- Assess how federal and state fund distribution formulas have 
impacted rural schools in their operations and course offerings* 

- Assess the Impact on rural schools of state school reform policies 
on course quality, diversity of course offerings, and student 
outcomes* 
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ATTACHMENT #2 



ROUND TWO DELPHI INSTRUMENT 
Areas for Research and Development on Rural Education 



In 1989, the Federal Interagency Committee on Education (PICE) Subcommit- 
tee on Rural Education examined the state of rural education research, 
drawing upon the knowledge of specialists within the federal government. 
From this examination, six topics emerged as priorities for furture 
research on rural education* These six priority topics, which are 
described in An Agenda for Research and Development on Rural Education , 
are: The Overall Effectiveness of Rural Schools; Curricular Provisions 
in Rural Schools; School and Community Partnerships on Behalf of Rural 
Schools; Human Resources for Rural Schools; Use of Technology in Rural 
Schools; and Financial Support and Governance for Rural Schools. For 
each topic, members of the Subcommittee generated sample research 
questions or areas of focus* 

Below are the six priority topics and the research questions you provided 
write-in answers for in Round One. As a participant in Round Two of the 
Delphi study, please read carefully each area presented under the mix 
.major topics and circle the degree to which you agree that this area is a 
priority: SA (Strongly Agree), A (Agree), N (Neutral), D (Disagree), and 
SD (Strongly Disagree). 

Following these six priority topics and new suggested research areas is a 
new major topic, Major Topic 7: Unique Aspects of Rural Communities. 
This topic was suggested in Round One as a major topic of research, not 
included in the Subcommittee's agenda, that should be examined. As a 
participant in Round Two of this Delphi study, please read carefully this 
major topic and ita description and circle the degree to which you agree 
that this topic is a priority. If you circle SA, A, or N, please provide 
suggested areas of research, within this topic, that should be examined. 



Major Topic 1: The Overall 
Effectiveness of Rural Schools 

(Researchera should clearly define the factors that describe and affect 
the rural community being studied, such ss geographic isolation; economy 
of scale; and variability in culture, economy, and social environment.) 

Area A: Assess the degree to which rural achools sre educsting students 
for participation in a national economy vs. s local economy. 

SA A N D SD 

Area B: Assess the students who lesve the rursl community snd either 
through sdvanced education or professional life become 
successful. 

SA A N D SD 
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Area C: Assess SEA role in rural education. 

SA A N D SD 

Area D: Assess teacher education institutions role in rural education. 

SA A N D SD 

Area E: Assess student expectations—view of the future. 

SA A N D SD 

Area F: Assess the ways in which rural school culture breaks down class 
distinctions or promotes increased cultural understanding. 

SA A N D SD 

Area 6: Contrast academic achievement of rural pupils taught in small 

classes against those in suburbs and urban schools with much 
larger class sizes. 

3A A N D SD 

Area H: Assess the role" of rural schools in an "integrated services" 
approach to meeting community needs. 

SA A N D SD 

Area I: Understand the change process and extent to which change 

initiated in one part of school can encourage change through- 
out school culture. 

SA A K D SD 



Major Topic 2: Curricular 
Provisions in Rural Schools 

(Serious curricular concerns have been raised over needs assessment, 
individualised instruction, design and implementation, cooperation with 
private sector development, and adult literacy improvement*) 

Area A: Assess satisfaction of students, teachers, administrators, 
parents, and community leaders with current curriculum and 
instruction. 

SA A N D SD 

Area B: Develop assessment strategies that link outcomes of experi- 
mental learning to conventional curricular. 

SA A N D SD 
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Area C: Assess ho* *tate and federal curriculum developed projects 
consider the needs of rural schools. 

SA A N D SD 

Area D: Identify strategies and programs from other countries that are 
transferable to American governance* 

SA A N D SD 

Major Topic 3: School and Community 
Partnerships on Behalf of Rural Schools 

(Research on school and community relationships should describe the 
environment within which learning occurs.) 

Area A: Assess level on parental and community involvement In rural area* 
SA A N D SD 

Major Topic 4: Human Resources 
for Rural Schools 

(Research on rural school personnel should be focused on recruitment, 
retention, professional development, administration, and supervision*) 

Area A: Identify effective beginnings teacher mentoring programs for 
rural schools* 

SA A N D SD 

Area B: Assess how business, indu ry, and community personnel can 

become teachers through alternative teacher education programs* 

SA A N D SD 

Major Topic 5: Use of Technology 
in Rural Schools 

(The learning outcomes achieved from each new technological advance need 
to be studied—individually and comparatively*) 

Area A: Assess importance of distance learning on rural students* 
SA A N D SD 

Area B: Conduct technology cost effectiveness* 

SA A N D SD 
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Area C: Assess level of private support for use of technology in rural 
schools. 

SA A N D SD 

Area D: Identify innovative low cost alternative programs to those 
delivered via telecommunications. 

SA A N D SD 



Major Topic 6: Financial Support 
and Governance for Rural Schools 

(Research should focus on the effects of school aid financial distri- 
bution formulas used by the states and by the federal government , the 
impact of school consolidation, and issues of education standards and 

quality.) 

Area A: Look at ways to equalise salary levels for teacher/adminis- 

trators in rural schools compared to benefits awarded teachers 
in large communities. 

SA A N D SD 

Area B: Explore legal and political implications for establishing local 
or regional foundations for seeking private funds and donations. 

SA A N D SD 

Area C: Compare and contrast the roles and strategies of rural Interest 
groups in the several school finance court cases in the states. 

SA A N D SD 



Major Topic 7: Unique Aspects 
of Rural Communities 

(More enthnographic studies at rural communities that cut across the 
topics above but focusing largely on the unique cultures, sociologies of 
particular rural settings in various geographic regions.) 

SA A N D SD 

If you circled SA, A, or N, what areas within this topic, Unique Aspects 
of Rural Communities, should be examined? (Please begin areas with a 
verb, as preceding examples.) 
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MEMORANDUM 



Improving Education Through 
ft 0 9 * * r c h and Dovolopmo nt 



TO: Rural Education SIG-Members 

FROM: John R. Sanders OtW 
DATE: November 4, 1992 Q 
SUBJECT: Round Three of Delphi Study 

Thank you for participating In Round Two of the Delphi study for the Rural 
Education Special Interest Group (RE/SIG). Of the 29 RE/SIG members* selected 
to participate In the study , 28 returned usable Instruments and the data were 
aggregated for Round Three. 

Of the original 24 Items on the Round Two Delphi Instrument, 16 were retained 
for Round Three. Items were retained If 90 percent of the respondents Strongly 
Agreed, Agreed, or were Neutral with/ toward the Item. 

The third round of the Delphi study contains the major topics and their 
suggested areas of research and development retained from Rounds One and Two 
(Attachment #1). Participants might recall that Round Two contained a seventh 
major topic (Unique Aspects of Rural Communities) and participants were asked 
to suggest areas of research and development necessary to Implement this topic. 
Based on data from Round Two, this major topic did not meet the 90 percent 
criterion set for the Delphi study; therefore, this topic and the suggested 
areas of research and development were not retained for Round Three. 

Round Three of the Delphi study consists of two parts: (A) prioritizing the 
six major topics and (B) developing a budget to address the six major topics. 

This mailing pertains to Part A. Please review the six major topics and their 
research and development areas In Attachment #1. Rank the six major topics 
from 1 to 6, with 1 being the highest and 6 the lowest, to reflect the order 
of priority In which you believe the topics should be researched. 

Once you have completed the Ranking Form (Attachment #2), mall It back to me 
at AEL by November 18, 1992. 



JRS:ksc 
Attachments 



♦Originally, a purposeful sample of 
study. One selected member did not 
participate In the study. A second 
survey for Round Two. 



30 RE/SIG members was selected for this 
return our calls and therefore did not 
selected RE/ SIC member did not return a 




Appatachia Educational Laboratory . A nonprofit corporation . An equal opportunity /affirmative action employer 
PC Box 1348. Charleston, WV 25325-7348 • Shipping: 1031 Quarrier Street. Charleston. WV 2530U2314 
Telephone 304/347-0400 • Toll free BOO/624-9120 • MX 304/347-04B7 
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Attachment #1 



ROUND THREE DELPHI INSTRUMENT 
Areas for Research and Development on Rural Education 



In 1989, the Federal Interagency Committee on Education (FICE) Subcommittee on 
Rural Education examined the state of rural education research, drawing upon 
the knowledge of specialists within the federal government. From this 
examination, six topics emerged as priorities for future research on rural 
education. These six priorities topics, which are described In An Agenda for 
Research and Development on Rural Education , are: The Overall Effectiveness 
of Rural Schools, Curricular Provisions In Rural Schools, School and Community 
Partnerships on Behalf of Rural Schools, Human Resources for Rural Schools, 
Use of Technology in Rural Schools , and Financial Support and Governance for 
Rural Schools. For each topic, members of the Subcommittee generated sample 
research questions or areas of focus* 

Below are the six major topics and the research and development areas retained 
from Rounds One and Two of the Delphi study* These topics and areas were 
retained If 90 percent of the respondents Strongly Agreed, Agreed, or were 
Neutral with/ toward the item* 

Round Three of the Delphi study consists of two parts: (A) prioritising the 
six major topics and (B) developing a budget to address the six major topics* 

As a participant In Part A of Round Three, please read carefully the following 
list detailing the six major topics and their suggested research and develop- 
ment areas necessary to Implement the topics* Then* rank the six major topics 
from 1 to 6, with 1 being the highest priority and 6 being the lowest priority, 
to reflect the relative urgency with which the major topics meed to be 
addressed* Use the enclosed Ranking Form (Attachment #2) to indicate your 
priority ranking* Once you have completed this form, mail it back to AEL In 
the self -addressed, stamped envelope by November 18, 1992* 



Major Topic 1: The Overall Effectiveness of Rural Schools 

Area A: Improve access to educational opportunity In Isolated rural 
communities* 

Area B: Identify the problems unique to the delivery of education In isolated 
rural communities in the following special populations: handicapped, 
disadvantaged, and gifted* 

Area C: Identify characteristics of effective rural schools* 

Area D: Conduct evaluation studies of student achievement in rural schools* 

Area E: Assess the federal role in rural education* 

Area F: Assess the degree to which rural schools are educating students for 
participation in a national economy vs. a local economy* 
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Area G: Assess SEA role in rural education* 

Area H: Assess teacher education institutions 9 role in rural education. 

Area I: Assess student expectations— view of % the future* 

Area J: Assess the ways in which rural school culture breaks down class 
distinctions or promotes increased cultural understanding. 

Area K: Assess the role of rural schools in an "Integrated services** approach 
to meeting community meeds. 

Area L: Understand the change process and extent to which change initiated in 
one part of school can encourage change throughout school culture* 

Major Topic 2: Curricular Provisions in Rural Schools 

Area A: Provide adult literacy improvement in isolated rural communities* 

Area B: Assess satisfaction of students , teachers, administrators, parents, 
and community leaders with current curriculum and instruction. 

Area C: Assess how state and federal curriculum development projects consider 
the needs of rural schools. 

Major Topic 3: School and Community Partnerships on Behalf of Rural Schools 

Area A: Identify effective school/community /private sector partnerships. 

Area B: Assess how federal/state/local policies are impacting rural schools 
and rural communities. 

Area C: Review legal procedures and issues pertaining to school and community 
partnerships on behalf of rural schools. 

Area D: Examine the factors of rural community economies that influence rural 
students 9 decisions to remain in school and graduate. 

Area E: Assess if Native American communities, or their learning environ- 
ments, differ from other rural communities. 

Area F: Assess the role of the rural school in promoting employ ability* 

Area G: Identify effective alternative schooling programs in the rural 
communities. 

Area H: Identify the social and cultural issues of isolated rural communities 
that Impact rural education* 

Area I: Assess level of parental and community involvement in rural area* 
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Major Topic 4: Human Resources for Rural Schools 

Area A: Identify successful strategies for the recruitment of qualified 
personnel to rural schools. 

Area B: Identify successful strategies for the retention of qualified 
personnel in rural schools • 

Area C: Identify strategies that have been successful for releasing rural 
teachers froa their classrooms for professional development. 

Area D: Identify successful leadership styles of effective rural school 
administrators . 

Area £: Identify the strategies used by administrators to comply with the 
state certification mandates. 

Area F: Assess the impact of recent state certification mandates on teacher 
availability in rural schools. 

Area G: Identify effective beginning teacher mentoring programs for rural 
schools. 



Major Topic 5: Use of Technology In Rural Schools 

Area A: Identify rural schools that have demonstrated effective use of 
advanced interactive instructional technology. 

Area B: Assess the impact of the advanced technology on rural school 
curriculums. 

Area C: Assess the effect of advanced technologies on traditional rural 
values of closeness, connection, or personal relationships in 
learning interactions. 

Area D: Assess the implications for instructional staff and support personnel 
vho are implementing advanced technology in rural school communities. 

Area £: Identify the staff development strategies that have been most success- 
ful in helping schools, teachers, and support personnel embrace and 
Integrate advanced technologies into their overall rural school system. 

Area F: Identify rural schools that successfully have implemented distance 
education via telecommunications. 

Area C: Conduct technology cost effectiveness studies* 

Area H: Assess level of private support for use of technology in rural schools* 

Area I: Identify innovative, low-cost alternative programs to those delivered 
via telecommunications. 
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Major Topic 6: Financial Support for Governance for Rural Schools 

Area A: Analyse the policies of school finance in rural communities. 

Area B: Identify alternatives to school consolidation for rural achool 
communities* 

Area C: Assess how federal and state fund distribution formulas have impacted 
rural schools In their operations and course offerings. 

Area D: Look at vays to equalize salary levels for teacher/administrators in 
rural schools compared to benefits awarded teachers In large 

communities* 

Area £: Compare and contrast the roles and strategies of rural Interest 
groups In the several school finance court cases in the states. 
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Attachment #2 



NREA/AEL DELPHI STUDY 
Part A: Ranking Form, November 1992 

As a participant In Part A of Round Three , please read carefully Attachment #1: 
Round Three Delphi Instrument* This details the six major topics and their 
suggested research and development areas for the NREA/AEL Delphi study* Rank 
the six major topics from 1 to 6, with 1 being the highest priority and 6 
being the lowest priority, to reflect the relative urgency with which the 
major topics need to be addressed* Write the number (1 through 6) on the 
space before each topic name* Use each number once* 

Major Topic 1: The Overall Effectiveness of Rural Schools 

Major Topic 2: Currlcular Provisions in Rural Schools 



Major Topic 3: School and Community Partnerships on Behalf of 
Rural Schools 



Major Topic 4: Human Resources for Rural Schools 

Major Topic 5: Use of Technology in Rural Schools 

Major Topic 6: Financial Support for Governance for Rural Schools 



Once you have completed this form, mail It back to John Sanders In the self- 
addressed, stamped envelope by November 18, 1992* 
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Improving Education Through 
Research and Development 



MEMORANDA 

TO: .Rural Education SIG Members 
FROMsvbftJltJohn R. Sanders 
DATE: ^ November 25, 1992 
SUBJECT: Round Three of Delphi Study 

Thank you for participating in Part A of Round Three of the Delphi study for 
the Rural Education SIG (RE/ SIG) group. This nailing pertains to Fart B: 
Developing a budget to address the six major topics. Assume that you are a 
director of a newly-formed national R&D center for rural educational 
research. In this role, you have a $5,000,000 budget to allocate for rural 
educational research for FY 93. This amount of funding for a federally- 
funded, national R&D center has been suggested recently by a prominent 
policymaker. 

Please review the six major topics and their suggested research and develop- 
ment areas in Enclosure #1. Next, locate the enclosed sheet of 10 blue 
stick-on dots. Think of each dot as 10 percent — $500,000 — of your budget. 
Tell us how this budget should be allocated among the six major topics. 
Complete the enclosed NREA/AEL Delphi Study Response Card and mail it back to 
me at AEL in the self-addressed, stamped envelope by December 9, 1992. 



JRSxksc 
Enclosures 



P.S. I'm still missing a few Part A Ranking Forms. For those of you 
who haven't returned your form yet, please do so by December 9. 
Thanks ! 




Appalachia Educational Laboratory • A nonprofit corporation « An equal opportunity /affirmative action employer 
PC Box 1348, Charleston, WV 25325-1348 ♦ Shipping: 1031 Quarrier Street. Charleston. WV 25301-2314 
Telephone 304/347-0400 ♦ Toll free 800/624-9120 • FAX 304/347-0487 
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Enclosure #1 



ROUND THREE DELPHI INSTRUMENT 
Areas for Research and Development on Rural Education 



In 1989 , the Federal Interagency Committee on Education (FICE) Subcommittee on 
Rural Education examined the state of rural education research, drawing upon 
the knowledge of specialists within the federal government. From this 
examination, six topics emerged as priorities for future research on rural 
education. These six priorities topics, which are described in An Agenda for 
Research and Development on Rural Education, are: The Overall Effectiveness 
of Rural Schools, Curricular Provisions in Rural Schools, School and Community 
Partnerships on Behalf of Rural Schools, Human Resources for Rural Schools, 
Use of Technology in Rural Schools, and Financial Support and Governance for 
Rural Schools. For each topic, members of the Subcommittee generated sample 
research questions or areas of focus. 

Below are the six major topics and the research and development areas retained 
from Rounds One and Two of the Delphi study. These topics and areas were 
retained if 90 percent of the respondents Strongly Agreed, Agreed, or were 
Neutral with/toward the Item. 

Round Three of the Delphi study consists of two parts: (A) prioritizing the 
six major topics and (B) developing a budget to address the six Major topics. 

As a participant in Part B of Round Three, we want you to assume the role of a 
director of a newly-formed national R&D center for rural educational 
research. In this role, you have a $5,000,000 budget to allocate for rural 
educational research for FY 93. 

Please read carefully the following list detailing the six major topics and 
their suggested research and development areas. Next, locate the enclosed 
sheet of 10 blue stick-on dots. Think of each dot as 10 percent — $500,000 — 
of your budget. 

We want you to tell us how this budget should be divided across these six 
topics. Indicate to us how much of your budget you are willing to allocate 
for research and development on each topic* Do this by placing as many dots 
as you would like in the area marked "Place dots here" for each major topic* 
Tou may use all 10 dots for one topic, or any combination of the 10 dots for 
all six topics. Not all dots need to be spent, nor all topics awarded research 
and development dollars. If you feel your budget would be best spent in 
"other" research areas, we have provided you with an opportunity to spend your 
money in an "other" category. 

As indicated above, each dot represents $500,000 of research and development 
dollars. This amount cannot be broken into smaller units. In other words, 
only whole dots will be counted toward the results for this round of the 
Delphi instrument. Also, please place the dots within the circles so we can 
determine how many dots are there. In other words, don 9 t stack all 10 dots on 
top of one another. 
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Once you have placed your dote In the appropriate box on the reaponae card, 
indicate , by number , how many dota you awarded to thia topic. Thia number 
goea in the area marked "Total t Awarded Thia ia juat a precaution in caae 
some of your dota cone off in the mail. Finally, provide ua with any 
qualifying comment a you feel might be neceaaary to aaaiat ua In understanding 
your allocation decisions. 

After you have finished dividing up your reaearch and development budget, 
place your completed reaponae card in the enclosed self -addressed, atamped 
envelope and mail it to me at AEL by December 9, 1992. 



Major Topic 1: The Overall Effectiveness of Rural Schools 

Area A: Improve access to educational opportunity in isolated rural 
communities. 

Area B: Identify the problems unique to the delivery of education in isolated 
rural communities in the following special populations: handicapped, 
disadvantaged, and gifted. 

Area C: Identify characteristics of effective rural schools. 

Area D: Conduct evaluation studies of student achievement in rural schools. 

Area E: Assess the federal role in rural education. 

Area F: Assess the degree to which rural schools are educating students for 
participation in a national economy vs. a local economy. 

Area G: Assess SEA role in rural education. 

Area H: Assess teacher education institutions 9 role in rural education. 

Area I: Assess 6tudent expectations— view of the future. 

Area J: Assess the ways in which rural school culture breaks down claas 
distinctions or promotes increaaed cultural understanding. 

Area K: Assess the role of rural schools in an "integrated services" approach 
to meeting community needs. 

Area L: Understand the change process and extent to which change initiated in 
one part of school can encourage change throughout achool culture* 



Major Topic 2: Curricular Provisions in Rural Schoola 

Area A: Provide adult literacy improvement in isolated rural communities. 

Area B: Assess satisfaction of students, teachers, adminiatrators, parents, 
and community leaders with current curriculum and instruction. 
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Area C: Assess how state and federal curriculum development projects consider 
the needs of rural schools* 



Major Topic 3; School and Community Partnerships on Behalf of Rural Schools 

Area A: Identify effective school/community /private sector partnerships* 

Area B: Assess how federal/state/local policies are impacting rural schools 
and rural communities* 

Area C: Review legal procedures and issues pertaining to school and community 
partnerships on behalf of rural schools* 

Area D: Examine the factors of rural community economies that influence rural 
students 9 decisions to remain in school and graduate* 

Area £: Assess if Native American communities, or their learning environ- 
ments, differ from other rural communities. 

Area F: Assess the role of the rural school in promoting employability* 

Area G: Identify effective alternative schooling programs in the rural 
communities. 

Area H: Identify the social and cultural issues of isolated rural communities 
that impact rural education. 

Area I: Assess level of parental and community involvement in rural area. 
Major Topic 4; Human Resources for Rural Schools 

Area A: Identify successful strategies for the recruitment of qualified 
personnel to rural schools. 

Area B: Identify successful strategies for the retention of qualified 
personnel in rural schools. 

Area C: Identify strategies that have been successful for releasing rural 
teachers from their classrooms for professional development* 

Area D: Identify successful leadership styles of effective rural school 
administrators * 

Area £: Identify the strategies used by administrators to comply with the 
state certification mandates* 

Area F: Assess the impact of recent state certification mandates on teacher 
availability in rural schools. 

Area G: Identify effective beginning teacher mentoring programs for rural 
schools* 
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Malor Topic 5: Use of Technology In Rural Schools 

Area A: Identify rural schools that have demonstrated effective use of 
Advanced interactive instructional technology. 

Area B: Assess the impact of the advanced technology on rural school 
curriculums* 

Area C: Assess the effect of advanced technologies on traditional rural 
values of closeness, connection, or personal relationships in 
learning interactions. 

Area D: Assess the implications for instructional staff and support personnel 
who are implementing advanced technology in rural school communities* 

Area £: Identify the staff development strategies that have been most success- 
ful in helping schools, teachers, and support personnel embrace and 
integrate advanced technologies into their overall rural school system* 

Area F: Identify rural schools that successfully have implemented distance 
education via telecommunications* 

Area G: Conduct technology cost effectiveness studies* 

Area H: Assess level of private support for use of technology in rural schools* 

Area I; Identify innovative, low-cost alternative programs to those delivered 
via telecommunications* 



Major Topic 6; Financial Support for Governance for Rural Schools 

Area A: Analyze the policies of school finance in rural communities* 

Area B: Identify alternatives to school consolidation for rural school 
communiti es* 

Area C: Assess how federal and state fund distribution formulas have impacted 
rural schools in their operations and course offerings* 

Area D: Look at ways to equalise salary levels for teacher/administrators 
in rural schools compared to salary levels for those In large 
communities* 

Area £: Compare and contrast the roles and strategies of rural Interest 
groups In the several school finance court cases in the states* 
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NREA/AEL Delphi Study 
Response Card 
November 1992 



Major Topic 1: The Overall Effectiveness of Rural Schools 



Place dots here Total $ Awarded Qualifying Comments 




Major Topic 2: Currlcular Provisions In Rural Schools 

Place dots here Total $ Awarded Qualifying Comments 




Major Topic 3: School and Community Partnerships on Behalf of Rural Schools 

Place dots here Total $ Awarded Qualifying Comments 




. dots X $500,000 



ERIC 



S3» please turn over e3> 
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Major Topic 4: Human Resources for Rural Schools 
Place dots here Total $ Awarded 




.dots X $500,000 



Qualifying Comments 



Major Topic 5: Use of Technology In Rural Schools 

Place dots here Total $ Awarded 




.dots X $500,000 



Qualifying Comments 



Major Topic 6: Financial Support and Governance for Rural Schools 

Place dots here Total $ Awarded Qualifying Comments 




. dots X $500,000 



Other 



Place dots here 



Total $ Awarded 




. dots X $500,000 



Qualifying Comments 



9 

ERIC 
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Commentaries 



72 



COMMENTARY 



I think of the study as attempting to accomplish two objectives: (1) compile a common cots of 
priority R&D areas for the nation's rural, small schools and (2) establish some priorities among those 
R&D areas. The study accomplished the first objective and did not provide conclusive evidence for the 
second objective. 

For the first obj ective, the study took the 19 R&D priority areas (organized around the same six 
topics) from the original 1991 agenda and succeeded in compiling an extended and consensus set of 47 
R&D areas, using the three-step Delphi methodology. I think thafs a real contribution. The process 
adds validity to the work of the committee that developed the original six topics and the priority areas 
under each. On the other hand, the study did not shed any light on the relative importance of these topics 
or ai^ in terms of either rank order of tq>ics(^ Perhaps 
participants cannot make the discriminations; maybe the R & D areas under the topics are so inter-related 
that grouping them into topics for rank ordering produces more noise than light. Maybe this is a lesser 
task anyway. All the six topics are important and, according to the participants of the study, worthy of 
support 

The study has validated the national R&D agenda for rural education. The issue of priority-setting, 
that is, which topic or area is more important or requires more immediate attention, is as much a regional 
and state concern as a national or federal one because building a national R&D agenda for rural schools 
is necessarily developmental, requiring interactions and accommodations between local and state needs 
and national interests. The challenge is here; the next step may be for the states to formulate their 
responses to it 

How should the findings be disseminated? What impact will the expanded agenda have on rural 
R & D? I think the revised list of R & D areas, as well as a brief description of the Delphi process, might 
be the subject of an ERIC Digest that can be distributed nationally through the Clearinghouse. This 
expanded R&D agenda will be helpful to state and regional deliberations on future R&D plans for rural 
and small schools. 

Stanley Chow, Far West Laboratory, San Francisco, CA 
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COMMENTARY 



There are many Americans who champion the quality of life provided in a rural setting. The inter- 
relatedness of the land and the community; the effects of the seasons and the local economy on the 
relationships between the people; and the absence of urban noise, clutter, and pollution all speak to a culture 
that values familiarity, personal connections, and privacy. The school is an important part of day-to-day 
activity in rural communities. Along with the library and the fire department, the schoolhouse serves to bring 
people together for the betterment of the town In doing so, the school exerts considerable influence on the 
development of the community . Whether graduates of rural schools choose to live in their home towns, or 
move to new places in the country or the world, the skills and knowledge that they accumulate from 
schooling during their childhood and adolescence have a tremendous effect on how each of them will spend 
their time as adits. 

It seems to me the results of the study presented in AEL Occasional Paper 25 dance around what nee& 
to be said clearly. How do students who have been educated in rural schools measure their success after they 
graduate from high school? Are they satisfied with their lives? Do they contribute to the community? Can 
they provide for themselves and their families financially? Are they lifelong learners? Do they seek answers 
to problems that may not be specific to only rural areas, but problems that snag the fabric of the country as 
a whole? I suspect that all educators could agree that these questions are at the heart of the matter. 

The most surprising finding from the study is the lack of emphasis on curricular provisions. 
Determining what content is taught, how that content is presented to students, and how well students meet 
the expectations established at the school site are the major responsibilities of teachers and administrators in 
rural schools. The usefulness of curricular blueprints and varied instructional approaches in creating 
meaningful learning environments for students needs to be continually examined If we are to facilitate the 
development of an informed citizenry with skills to cope with the changes that will surely accompany the 
arrival of the 21st century, this task should be at the forefront of our coordinated efforts. 

Creating opportunities for rural educators to critically assess what is actually taking place within the 
walls of the schoolhouse with regard to curriculum, instruction, and student learning is not an empty exercise. 
Some questions that might be explored follow. Do our teachers and students know how to learn? Do they 
have access to adequate resources? Do parents understand what the school is doing? Does the curriculum 
match the needs of the community? How can instructional time be best organized? Is technology a 
successful instructional strategy? What about cross-age groupings? Internships with local businesses? Are 
learning experiences designed to promote the use of advanced thinking skills? 

Typically, the time necessary to thoughtfully answer these kinds of questions is not allocated regularly. 
By delving into the areas of curriculum and instruction, many issues surface that require teachers, parents, and 
community members to question what they believe to be important about educating students so that they can 
successfully contribute to the future of the community. Conducting focus groups, surveys, long-term staff 
development, case studies, and assessing student progress through the use of multiple measures can lead to 
meeting outcomes in the process of developing rural schools and communities. 

Placing the curriculum at the center of the rural research and development agenda could enhance a more 
systemic approach to the overall improvement of rural education. All six of the recommended topics are linked, 
although the webbing may be complex, to the productivity aixl success of the student Keeping curriculum as the 
focal point of our thinking may assist us with the formidable task of consolidating the range of topics presented 
by FICE and the Rural Education Special Interest Group. In doing so, some of the generalities present in the 
conclusions of AEL Occasional Paper 35 may be delineated further, preventing the loss of valuable time, energy, 
and resources that could be better used in advancing the rural education research and development menu. 

Gail Gordon, RMC Research Corporation, Baltimore, MD 
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COMMENTARY 



This study is a significant and useful effort for pushing forward the research and development 
related to rural schools. Following the FICEs initial work, the project tried to clarify important issues 
facing rural schools. One of the project's distinctive features is that information was collected based on 
carefully weired judgments of researchers who are active in the field The other important feature of 
this study is that the project was conducted by AEL, which, located in the hinterland of the rural 
Appalachia region, has been long focusing its attention on rural education. 

Important Findings 

The most salient finding of the study is the panel members' high emphasis on the effectiveness of 
rural schools. Indeed, the quality of educational institutions is the key concern for both rural communities 
and the nation as a whole (see Special Study Panel on Education Indicators, 1991). School effectiveness 
is the main theme of this study, and all the other major topics (curriculum, school-community partnerships, 
staffing, technology, and financing) actually are subdimensions of, or instruments to realize, school 
effectiveness. My concern is not so much about the priority rankings of the topics, but the balance of the 
issue coverage and the logical and practical connections across the research/development tasks. With such 
a concern, it makes sense to me that the panel had a strong consensus on the need for research on the 
overall issue of school effectiveness, but expressed more diverse judgments on the other main topics. 

The results from Round Two ofthe study seem most important The topics identified here are either 
practically pressing or scholarly interesting or both: integrated services; SEA role in rural education (I 
would add the roles of rural districts and school buildings in policymaking); higher education role; 
students' expectation, particularly their views of the future; the extent to which rural schooling works to 
enable rural students to fit into local or national economies; and rural school environment's role in altering 
class differentiation. Using NCES data, I've conceptualized an analysis on the changing policymaking 
roles of school districts in rural areas. In the current restructuring process, the federal and state influences 
are drastically increasing (with a host of powerful mechanisms such as resources allocation and incentive 
development, mandates and regulations, and guidance and standardization); while local school-based 
management also claims large chunks of decisionmaking power (Elmore, 1993). In the middle of such 
a simultaneous centralization and decentralization, what is the school district role in policymaking? Great 
discrepancy can be expected of rural districts' responses to the changes, in contrast to the large urban 
districts that function as de facto SEAs. 

The proposed research on rural students 1 expectations of the future and students' participation in local 
versus national labor markets is closely tied together. Both imply a fundamental paradox feeing rural 
communities and schools. That is, with rural communities' meager resources and desperate needs for the 
educated, rurai schools are training rural youth with urban-oriental skills so that they can leave their native 
communities for the urban labor market (DeYoung, 1991). To deal with this issue in research, both large 
structural factors and local process factors should be considered For instance, die local economy's 
position in the national/global economy and school programs/cumculum can affect students' expectation 
for outmigration or remaining. Liberal arts proponents say that humanities educatiai that incorporates the 
local culture can help strengthen rural youths' attachment to the community. Vocational/technical 
educators see occupational skills as essential for rural kids to get jobs and to ultimately revitalize rural 
economies. Empirical examinations to both claims are badly needed. 
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Hie other interesting idea suggested by the panel is the rural culture's function in reducing class 
distinction. If I understand correctly, rural culture sanctions close interpersonal relations that are crucial 
for teaching and learning. Intimate interaction among teachers, students, and family fosters a school 
climate that reduces the detrimental effect of poverty. Particularly, such a climate may reduce the 
differentiation in student outcomes related to SES. 

Major Topic 2 (curriculum) enlists issues on the impact of program/curriculum on psychosocial 
outcomes of rural schooling. How do state-guided or state-standardized curricula meet the needs of the 
local people, and contribute to community development? What are and should be the input of local folks 
to the curriculum/instruction so that learning can strengthen the local social integration and cultivate 
youngsters 1 sense of meaning in rural life? The issue, linking to the debate on liberal arts versus 
occupational education, is of high-stake for both policymaking and pedagogy. 

Major Topic 3 refers to a critical matter in rural education: parental and community involvement 
in school. What are the ways by which rural families participate in kids' schooling? How do parents help 
kids learn? What are the activities parents are involved with in school? What are the consequences of 
such help and participation to student outcomes? And how can we improve parental and community 
participation? Systematic efforts are needed to address these questions. 

In terms of Major Topic 6, financial support, baseline data are needed to provide national, state, and 
local pictures about financial equity. We need comparisons across locales and administrative levels. Such 
comparisons should be done in an ongoing process, perhaps every three or five years on the national level, 
and every year on the state or local level. Fair and effective decisions can be made based on valid and 
precise comparative analyses. 

An Additional Research Issue 

Early childhood services and education are recognized as a critical approach to handling the 
problems related to high-risk students. The notion of "ready to learn" is a powerful statement of a national 
education goal. These issues were ignored in this study. The matter seems especially relevant to rural 
schooling because early childhood services seem to be in acute shortage in rural areas. Research tasks 
may include: (1) collecting and interpreting baseline data on maternal health, nutrition, prenatal health, 
infant health, and preschool health and development; (2) exploring rural family environment effects on 
early childhood development; (3) studying nonparental child care services distribution and utilization in 
rural areas; (4) identifying effective parent education strategies for improving early childhood parenting 
skills; (5) assessing Headstart and other programs In rural settings; and (6) examining the impact of early 
childhood programs to later schooling. 

Gary Huang, National Center for Education Statistics, Washington, DC 
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COMMENTARY 



Friorities for Research and Development on Rural, Small Schools, a study conducted by the 
Appalachia Educational Laboratory with selected members of the Rural Education SIG of AERA, provides 
a rich menu of researchable issues on rural education 

In my present research study on defining the characteristics of effective elementary schools in West 
Virginia, I had not included the subdivision of characteristics of effective rural schools or the subdivision 
of student achievement in rural areas. The list of research topics provided by the AEL Delphi study 
became a strong reminder of important rural issues that needed to be investigated and that could h; 
included in my present work. All I needed was the reminder and a rural code to identify rural schocis 
in my data set to include the two important rural education areas in my present research study. 

To me, the most interesting finding of this study was that a group of 28 rural researchers across the 
United States took the time to complete the surveys and provide the public with such a rich anay of topics 
to explore. Because of the selecticni process provided by the Delphi study, I feel confident that the topics 
are worthy of investigation. 

The most important finding is the potential utility of the research and development menu for 
researchers, graduate students, and policymakers. The menu could be used as a source of research topics 
for graduate students, as a reminder of important niral areas to indude in pre 

for researchers, and as a gauge for measuring the importance of rural issues by policymakers. I feel that 
it will be beneficial to my work to review the list of topics each time I start a new research project for 
possible inclusion of items and as a refresher of potential areas of research. For greatest utility, I suggest 
that the agenda be published in a form that can be attached to a bulletin board or retrieved with little or 
no effort 

For the research and development menu to have the maximum impact on rural, small schools, the 
menu should be sent to the members of AERA, the National Rural Education Association, and to graduate 
schools across the United States as suggested topics for potential research and graduate studies. 

For additional benefit to the rural areas, one topic could be examined by each state and the results 
compiled to form a national overview. For example, under Major Topic 1 : The Overall Effectiveness of 
Rural Schools, a specific area such as the federal role in rural education could be explored across the 
United States. I would think that this type of information would be of great interest to state and federal 
policymakers. 

If researchers across the United States start including an item from the research and development 
menu in their research project, or select an item as a maj or topic, then v*e may sec an increased awareness 
of rural education issues across the nation The research and development menu will plant the seed in the 
researcher's mind and will encourage research in an area that may have been overlooked 

I commend AEL and the group of 28 rural researchers for their efforts in providing such a rich array 
of rural research topics to the research community. I feel your efforts will have an important impact on 
future research on rural education 

Mary Hugjies, West Virginia Education Fund, Charleston, WV 
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What can we learn from the Delphi study of priorities for research and development in rural 
education? One thing we learn is that the Federal Interagency Committee on Education (FICE) 
subcommittee on rural education's 1989 research agenda still holds up in 1993. The Delphi study leaves 
intact all six original major topics. No others met the criteria for inclusion This makes a good case for 
considering these the important areas for research and development in rural education 

A closer look at the specific items under several of the major topical areas strengthens the case for 
the FICE agenda. For example, two topical areas emerged virtually unchanged These were school and 
community partnerships (Major Topic 3) and human resources for rural schools (Major Topic 4). The 
partnership topic pulled in fully 89 percent of the specific items from the original FICE list and the human 
resources topic retained a healthy 86 percent By the end of Round Two, the Delphi study respondents 
had added only one new item to each of these topical areas. Two other major topical areas retained more 
than half of the ori ginal items. These have to do with the use of technology in rural schools (Maj or Topic 
5) and the overall effectiveness of rural schools (Major Topic 1). 

Further examination of the details, however, suggests that the Delphi respondents changed the FICE 
agenda to a greater extent than may be obvious at first The amount of change is relatively clear in the 
areas of auricular provisions (Major Topic 2) and financial support for governance (Major Topic 6). In 
these topical areas, the respondents retained the fewest items from the 1989 FICE list and added a 
relatively high proportion of new items. (Only one or 20 percent of the auricular topic's original five 
items made the final list; the two additions from Delphi respondents made up 67 percent of the final set) 

Substantial recasting of the FICE agenda by the Delphi respondents can also be seen in the area of 
overall effectiveness of rural schools (Major Topic 1). Although over 60 percent of the FICE items under 
this heading made the Delphi study's final list, they were outnumbered by new additions from the Delphi 
respondents. The respondents tended to drop items relating to programs in rural schools and tended to 
add items about the connection of rural schools to other institutions (e.g., the state, the community, the 
economy) and the nature of culture in rural schools. 

Other analyses would reveal more about the nature of the subtle shift in priority or in the meaning 
of the priorities from 1989 to 1993. One such analysis might be to compare the substance of the items 
retained and those dropped from the FICE items. Another might be comparing the topics and items that 
the Delphi respondents proposed and subsequently retained or dropped 

Still more mi^it be gained by a discussion among the Delphi respondents. A face-to-face discussion 
would not only allow them to explain the mismatch between their ratings of priority and assignment of 
resources, but could also give them an opportunity to attenpt to reach consensus on the intellectual and 
material priority of the topics within the research and policy community. The Delphi study gives us an 
idea of the amount of convergence of their ideas; having them argue out their views might produce a 
powerful consensus about these (or maybe even other) ideas. 

How does the Delphi study hold up against the views of rural education practitioners? A study of 
rural superintendents' perceptions of critical issues in rural education, which was conducted in Maryland 
about the same time, provides a comparison. Informants in the Maryland study included the members of 
the Rural Assistance Council (RAQ, which is composed of superintendents from Maryland's seven low- 
wealth school systems and the director and codirector of a nine-system rural consortium The school 
systems in the RAC are located in the extreme western part of the state, an Appalachian region; and in 
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the south central part of the state and on the eastern shore of the Chesapeake, both regions of fanning and 
water products. County-based, these school systems enroll from 5,000 K-12 students in the smallest to 
15,000 in the largest 

The RAC study was conducted by Research for Better Schools and involved individual telephone 
interviews of RAC members (N = 9). Responses are presented in decreasing order of frequency. 

• Most RAC members (78 percent) identified funding and equity as a critical issue. In the Delphi 
study, the equivalent topic (Major Topic 6) ranked third, but was judged to have relatively 
modest salience. 

• Most (78 percent) also identified lowest implementation as a top priority. Implementing 
promising practices within severe fiscal constraints was the issue, not access to ideas nor 
knowledge about promising practices. This also links most closely to the financial support topic 
(Major Topic 6). However, it is obliquely connected to items that refer to effective programs, 
especially if the judgment of their effectiveness includes a cost and/or resource criterion. 
Examples might be improving access to educational opportunity in Major Topic 1, assessing 
how state and federal curriculum development projects consider rural needs in Major Topic 2, 
identifying effective alternative schooling programs under Major Topic 3, and so oa 

• Just over half (56 percent) of RAC members cited family and community involvement in 
education as a critical issue. They were interested both in how to involve parents in school 
improvement and in how to generate support for education in their communities. Another 
aspect of this issue was how to promote school-business partnerships in the face of high 
unemployment and few r local industries. This set of issues relates clearly to the Delphi study s 
Major Topic 3 on school and community partnerships. 

• Slightly under half (44 percent) ofRAC members identified school improvement and reform as 
the next most pressing issues in rural education. Of particular interest was how to help staff and 
community welcome and institutionalize change. Understanding change was a small part of the 
Delphi study's overall effectiveness of rural schools (Major Topic 1). 

• One-third (33 percent) of RAC members indicated that staff development was a critical issue. 
Like low-cost implementation, this issue had implications in other topical areas. Specific 
aspects of this issue had to do with high-impact content and methods to renew a veteran 
teaching force, and developing staff adequately for the continuous stream of state mandates. 
This issue is part of Major Topic 4, human resources for rural schools, in the Delphi study. 

• Integrated services also concerned one-third (33 percent) of RAC members. This correlates to 
a portion of the Delphi study's Major Topic 1, overall effectiveness of rural schools. 

Overall, the RAC members shared some of the Delphi respondents' issues, but the two agendas differ 
in emphasis. The RAC members want to know how to do what they have to do. They are executives and 
theirs is an agenda for action. Specifically, they want to know how to make do with few resources. 

The similarities among the FICE, Delphi, and die RAC lists suggest fundamental agreement about 
research and development priorities in rural education. The differences suggest the need for dialogue- 
among the Delphi researchers and policymakers, as well as between them and practitioners. And if a 
national research and development center for rural education were to be established, it might be wise to 
add that dialogue to its list of priorities. 

Gail Meister, Research for Better Schools, Philadelphia, PA 
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